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Why GAO Did This Study 

Reducing the incidence of sexually 
transmitted diseases and 
unintended pregnancies is one 
objective of the Department of 
Health and Human Services (HHS). 
HHS provides funding to states and 
organizations that provide 
abstinence-until-marriage 
education as one approach to 
address this objective. 

GAO was asked to describe the 
oversight of federally funded 
abstinence-until-marriage 
education programs. GAO is 
reporting on (1) efforts by HHS and 
states to assess the scientific 
accuracy of materials used in these 
programs and (2) efforts by HHS, 
states, and researchers to assess 
the effectiveness of these 
programs. GAO reviewed 
documents and interviewed HHS 
officials in the Administration for 
Children and Families (ACF) and 
the Office of Population Affairs 
(OPA) that award grants for these 
programs. 



What GAO Recommends 



To address concerns about the 
scientific accuracy of materials 
used in abstinence-until-marriage 
education programs, GAO 
recommends that the Secretary of 
HHS develop procedures to help 
assure the accuracy of such 
materials used in programs 
administered by ACF. HHS agreed 
to consider this recommendation. 
HHS also provided information on 
steps it takes to assure accuracy, 
which we have incorporated into 
the report, as appropriate. 

www.gao.gov/cgi-bin/getrpt7GAO-07-87. 

To view the full product, including the scope 
and methodology, click on the link above. 

For more information, contact Marcia Crosse 
at (202) 512-7119 orcrossem@gao.gov. 



What GAO Found 

Efforts by HHS and states to assess the scientific accuracy of materials used 
in abstinence-until-marriage education programs have been limited. This is 
because HHS’s ACF — which awards grants to two programs that account for 
the largest portion of federal spending on abstinence-until-marriage 
education — does not review its grantees’ education materials for scientific 
accuracy and does not require grantees of either program to review their 
own materials for scientific accuracy. In contrast, OPA does review the 
scientific accuracy of grantees’ proposed educational materials. In addition, 
not all states that receive funding from ACF have chosen to review their 
program materials for scientific accuracy. In particular, 5 of the 10 states 
that GAO contacted conduct such reviews. Officials from these states 
reported using a variety of approaches in their reviews. While the extent to 
which federally funded abstinence-until-marriage education materials are 
inaccurate is not known, in the course of their reviews OPA and some states 
reported that they have found inaccuracies in abstinence-until-marriage 
education materials. For example, one state official described an instance in 
which abstinence-until-marriage materials incorrectly suggested that HIV 
can pass through condoms because the latex used in condoms is porous. 

HHS, states, and researchers have made a variety of efforts to assess the 
effectiveness of abstinence-until-marriage education programs; however, a 
number of factors limit the conclusions that can be drawn about the 
effectiveness of abstinence-until-marriage education programs. ACF and 
OPA have required their grantees to report on various outcomes that the 
agencies use to measure the effectiveness of grantees’ abstinence-until- 
marriage education programs. In addition, 6 of the 10 states in GAO’s review 
have worked with third-party evaluators to assess the effectiveness of 
abstinence-until-marriage education programs in their states. Several factors, 
however, limit the conclusions that can be drawn about the effectiveness of 
abstinence-until-marriage education programs. Most of the efforts to 
evaluate the effectiveness of abstinence-until-marriage education programs 
included in GAO’s review have not met certain minimum scientific criteria — 
such as random assignment of participants and sufficient follow-up periods 
and sample sizes — that experts have concluded are necessary in order for 
assessments of program effectiveness to be scientifically valid, in part 
because such designs can be expensive and time-consuming to carry out. In 
addition, the results of efforts that meet the criteria of a scientifically valid 
assessment have varied and two key studies funded by HHS that meet these 
criteria have not yet been completed. When completed, these HHS-funded 
studies may add substantively to the body of research on the effectiveness of 
abstinence-until-marriage education programs. 
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United States Government Accountability Office 
Washington, DC 20548 



October 3, 2006 
Congressional Requesters 

Preventing sexually transmitted diseases (STD) and unintended 
pregnancies among adolescents is an important public health challenge. 
Although pregnancy and birth rates among female adolescents in the 
United States have been declining since the early 1990s, the rates continue 
to be high when compared with those in other industrialized nations. The 
Centers for Disease Control and Prevention (CDC) reports that about 
141,000 children were bom to girls 17 years old and younger in the United 
States in 2003. CDC also reports that STDs disproportionately affect 
adolescents, with adolescents and young adults ages 15 to 24 acquiring 
almost half of the estimated 19 million new infections each year. 

Reducing the incidence of STDs and unintended pregnancies among 
adolescents is an important objective for the Department of Health and 
Human Services (HHS), which identifies as one of its goals the need to 
reduce major threats to the health and well-being of Americans. Among 
the efforts it supports to reduce the incidence of STDs and unintended 
pregnancies among adolescents, HHS funds abstinence-until-marriage 
education programs. Abstinence-until-marriage education programs, also 
referred to as abstinence-only education programs, teach adolescents to 
abstain from sexual activity until marriage in order to avoid risks of 
unintended pregnancy, STDs, and related health problems. 1 The content of 
federally funded abstinence-until-marriage programs is required to be 
consistent with several principles, such as teaching that a mutually faithful 
monogamous relationship in the context of marriage is the expected 
standard of human sexual activity, and that abstinence from sexual 
activity is the only certain way to avoid STDs. Abstinence-until-marriage 
education programs are delivered by a variety of entities, including 
schools, human service agencies, faith-based organizations, youth 
development groups, and pregnancy crisis centers. Instructors can 
incorporate a variety of educational materials into their abstinence-until- 
marriage education programs, including textbooks, student manuals, 
brochures, slide presentations, and videos. 



Abstinence-until-marriage education programs also support HHS’s objective to promote 
family formation and healthy marriages. 
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The three main federally funded abstinence-until-marriage programs are 
the Abstinence Education Program (State Program), which is administered 
by HHS’s Administration for Children and Families (ACF); the Community- 
Based Abstinence Education Program (Community-Based Program), 
which is also administered by ACF; and the Adolescent Family Life (AFL) 
Program, which is administered by HHS’s Office of Population Affairs 
(OPA) within the Office of Public Health and Science. Funding provided 
by HHS for the three abstinence-until-marriage programs increased from 
about $73 million in fiscal year 2001 to about $158 million in fiscal year 
2005. 

Recent studies have raised concerns about the accuracy of educational 
materials used in abstinence-until-marriage education programs and about 
the effectiveness of these programs. These studies have reported that 
some of the materials used in abstinence-until-marriage education 
programs contain, for example, scientifically inaccurate information about 
anatomy and physiology as they relate to reproductive health as well as 
misleading information about contraceptive failure rates and STDs. 2 State 
and federal agencies have also documented inaccuracies in abstinence- 
until-marriage educational materials. Further, studies examining the 
effectiveness of these programs have reported varied results. For example, 
some researchers have reported that abstinence-until-marriage education 
programs have resulted in adolescents reporting less frequent sexual 
intercourse or fewer sexual partners, while other researchers have 
reported that these types of programs did not affect the frequency of 
sexual intercourse or were ineffective in delaying the initiation of sexual 
intercourse. 3 

You asked us to describe certain aspects of the oversight of federally 
funded abstinence-until-marriage education programs. Our objectives 
were to report on (1) efforts by HHS and states to assess the scientific 



2 See K. L. Wilson et al., “A Review of 21 Curricula for Abstinence-Only-Until-Marriage 
Programs,” The Journal of School Health, vol. 75, no. 3 (2005), and The Content of 
Federally Funded Abstinence-Only Education Programs, United States House of 
Representatives, Committee on Government Reform-Minority Staff, Special Investigations 
Division (2004). 

3 See, for example, E. A. Borawski et al., “Effectiveness of Abstinence-only Intervention in 
Middle School Teens,” American Journal of Health Behavior, vol. 29, no. 5 (2005), and 
J. B. Jemmott III, L. S. Jemmott, and G. T. Fong, “Abstinence and Safer Sex HIV Risk- 
Reduction Interventions for African American Adolescents: A Randomized Controlled 
Trial,” Journal of the American Medical Association, vol. 279, no. 19 (1998). 
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accuracy of materials used in abstinence-until-marriage education 
programs and (2) efforts by HHS, states, and researchers to assess the 
effectiveness of abstinence-until-marriage education programs. You also 
asked us to describe how HHS selected a contractor for the abstinence- 
until-marriage technical assistance contract that was awarded in 
September 2002. This information is provided in appendix I. 

To describe the efforts by HHS and states to assess the scientific accuracy 
of materials used in abstinence-until-marriage education programs, we 
reviewed published reports, program announcements, Federal Register 
notices, agency Web sites, and other documents related to abstinence- 
until-marriage education. We focused our review on efforts related to the 
three main federally funded abstinence-until-marriage education programs 
administered by HHS, as well as efforts to review the accuracy of scientific 
facts included in abstinence-until-marriage education materials. We did 
not assess the criteria used to determine the scientific accuracy of 
education materials or the quality of the reviews. We interviewed officials 
from ACF, the Health Resources and Services Administration (HRSA), 
OPA, and CDC. We also interviewed officials from the 10 states that 
received the largest share of federal funding (together accounting for 
51 percent of the total funding in fiscal year 2005) through the State 
Program for abstinence-until-marriage education. 4 

To describe efforts by HHS, states, and researchers to assess the 
effectiveness of abstinence-until-marriage education programs, we 
focused on efforts that examined the extent to which these programs 
achieved their program goals. In general, these goals include teaching 
adolescents to abstain from sexual activity until marriage in order to avoid 
unintended pregnancies, STDs, and related health problems. As part of our 
review, we compared these efforts to the design characteristics that 
experts have identified as important for a scientifically valid study of 



4 The 10 states that received the largest share of funding in fiscal year 2005 through the 
State Program were Arizona, Florida, Georgia, Illinois, Louisiana, Michigan, New York, 
North Carolina, Ohio, and Texas. 
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program effectiveness. 5 We reviewed journal articles and other published 
reports, agency budget submissions, program announcements, agency and 
grantee performance reports, Federal Register notices, agency Web sites, 
and other documents related to abstinence-until-marriage education. (For 
a more detailed description of our literature review methodology, see 
app. II). We also interviewed officials from ACF, HRSA, OPA, CDC, the 
National Institutes of Health (NIH), the Office of the Assistant Secretary 
for Planning and Evaluation (ASPE), and 10 states that received the largest 
share of federal funding for abstinence-only education through the State 
Program in fiscal year 2005. We also interviewed individuals from the 
National Campaign to Prevent Teen Pregnancy, The Brookings Institution, 
ETR Associates, The Heritage Foundation, and Advocates for Youth, and 
researchers from Case Western Reserve University and Columbia 
University to obtain general information regarding the state of the 
research on abstinence-until-marriage education. We focused our review 
on efforts to assess the scientific accuracy of materials and the 
effectiveness of the programs during fiscal year 2006, and also reviewed 
the administration of the programs back to fiscal year 2001. We also 
attended conferences organized by ACF and OPA to learn about training 
that is provided to grantees on scientific accuracy and program 
evaluations. 

To describe how HHS selected a contractor for the abstinence-until- 
marriage technical assistance contract that was awarded in September 
2002, we reviewed the Request for Proposals and other related contract 
documents. We also interviewed officials at HRSA, ACF, and the National 
Abstinence Clearinghouse about the technical assistance contract. We 
performed our work from October 2005 through September 2006 in 
accordance with generally accepted government auditing standards. 



5 See Douglas Kirby, Emerging Answers: Research Findings on Programs to Reduce Teen 
Pregnancy (Washington, D.C.: National Campaign to Prevent Teen Pregnancy, 2001). The 
experts identifying the design characteristics of a scientifically valid study for the National 
Campaign to Prevent Teen Pregnancy were drawn from institutions that include the 
National Institutes of Health, the Medical Institute for Sexual Health, the Alan Guttmacher 
Institute, the Institute for Research and Evaluation, and various universities. See David 
Satcher, The National Consensus Process on Sexual Health and Responsible Sexual 
Behavior: Interim. Report (Atlanta: Morehouse School of Medicine, 2006). The panel 
convened by David Satcher included experts from a variety of organizations, including the 
Medical Institute for Sexual Health, the Alan Guttmacher Institute, and the American 
Academy of Pediatrics. In addition, characteristics of a scientifically valid study have been 
identified by other experts in the field of evaluation research. For example, see Carol H. 
Weiss, Evaluation (Upper Saddle River: Prentice Hall, 1998). 
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Results in Brief 



Efforts by HHS and states to assess the scientific accuracy of materials 
used in abstinence-until-marriage education programs have been limited. 
This is because ACF — which awards grants through two programs that 
account for the largest portion of federal spending on abstinence-until- 
marriage education — does not review its grantees’ education materials for 
scientific accuracy and does not require grantees of either program to 
review their own materials for scientific accuracy. In addition, not all 
states that receive funding through ACF’s State Program have chosen to 
review their program materials for scientific accuracy. In particular, 5 of 
the 10 states in our review conduct such reviews. Officials from these 
states reported using a variety of approaches in their reviews. In contrast, 
OPA does review the scientific accuracy of AFL grantees’ proposed 
educational materials and any inaccuracies found must be corrected 
before the materials can be used. While the extent to which federally 
funded abstinence-until-marriage education materials are inaccurate is not 
known, in the course of their reviews OPA and some states reported that 
they have found some inaccuracies in abstinence-until-marriage education 
materials. For example, OPA has required that a grantee correct several 
statements in a true/false quiz — including statements about STDs and 
condom use — in order for the quiz to be approved for use in its 
curriculum. In addition, one state official described an instance in which 
abstinence-until-marriage materials incorrectly suggested that HIV can 
pass through condoms because the latex used in condoms is porous. 

HHS, states, and researchers have made a variety of efforts to assess the 
effectiveness of abstinence-until-marriage education programs; however, a 
number of factors limit the conclusions that can be drawn about the 
effectiveness of abstinence-until-marriage education programs. To assess 
the effectiveness of their abstinence-until-marriage education programs, 
ACF and OPA have required their grantees to report on various outcomes 
that the agencies use to measure the effectiveness of grantees’ abstinence- 
until-marriage education programs. For example, as of fiscal year 2006, 
states that receive funding through the State Program are required to 
report annually on four measures of the prevalence of adolescent sexual 
behavior in their state, such as the rate of pregnancy among adolescents 
aged 15 to 17 years. To assess the effectiveness of both its State and 
Community-Based Programs, ACF also analyzes trends in adolescent 
behavior, as reflected in national data on birth rates among teens and the 
proportion of surveyed high school students reporting that they have had 
sexual intercourse. OPA requires grantees of the AFL Program to develop 
and report on outcome measures that demonstrate the extent to which 
grantees’ programs are having an effect on program participants. In 
addition, other HHS agencies and offices — ASPE, CDC and NIH — are 



Page 5 



GAO-07-87 Abstinence Education 



making efforts to assess the effectiveness of abstinence-until-marriage 
education programs. Further, 6 of the 10 states in our review that receive 
funding through the State Program have worked with third-party 
evaluators to assess the effectiveness of abstinence-until-marriage 
education programs in their states. Several factors, however, limit the 
conclusions that can be drawn about the effectiveness of abstinence-until- 
marriage education programs. Most of the efforts to evaluate the 
effectiveness of abstinence-until-marriage education programs that we 
describe in our review have not met certain minimum criteria — such as 
random assignment of participants and sufficient follow-up periods and 
sample sizes — that experts have concluded are necessary in order for 
assessments of program effectiveness to be scientifically valid, in part 
because such designs can be expensive and time-consuming to carry out. 
In addition, the results of efforts that meet the criteria of a scientifically 
valid assessment have varied and two key studies funded by HHS that 
meet these criteria have not yet been completed. When completed, these 
HHS-funded studies may add substantively to the body of research on the 
effectiveness of abstinence-until-marriage education programs. 

To address concerns about the scientific accuracy of materials used in 
abstinence-until-marriage education programs, we recommend that the 
Secretary of HHS develop procedures to help assure the accuracy of such 
materials used in the State and Community-Based Programs. To help 
provide such assurance, the Secretary could consider alternatives such as 
(1) extending the approach currently used by OPA to review the scientific 
accuracy of the factual statements included in abstinence-until-marriage 
education to materials used by grantees of ACF’s Community-Based 
Program and requiring grantees of ACF’s State Program to conduct such 
reviews or (2) requiring grantees of both programs to sign written 
assurances in their grant applications that the materials they propose 
using are accurate. 

In commenting on a draft of this report, HHS agreed to consider requiring 
grantees of both ACF programs to sign written assurances in grant 
applications that the materials they use are accurate. In addition, HHS 
noted that all federal grant applicants attest on a standard form that 
information in their applications is correct. However, it is not clear that 
this serves the purpose of assuring the scientific accuracy of the 
educational materials. Further, the curricula to be used are not required to 
be included with states’ applications. HHS’s written comments also stated 
that ACF requires that the Community-Based Program curricula conform 
to standards that are grounded in scientific literature by requiring certain 
types of information. However, the inclusion of certain types of 
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information does not necessarily ensure the accuracy of the scientific facts 
included in the abstinence-until-marriage materials. In addition, HHS 
noted in its written comments that we did not define the term scientific 
accuracy and stated that it disagreed with certain findings of the report 
because it was difficult to precisely determine the criteria employed by 
GAO in making the recommendation as to scientific accuracy. However, 
the objective of our work was to focus on efforts by HHS and states to 
review the accuracy of scientific facts included in abstinence-until- 
marriage education materials and not to perform an independent 
assessment of the criteria used or the quality of the reviews. With regard to 
effectiveness, HHS agreed that it may be too soon to draw conclusions 
about the effectiveness of ACF’s and OPA’s programs. 



Background 



Recent statistics from CDC show that many high school students engage in 
sexual behavior that places them at risk for unintended pregnancy and 
STDs. In 2005, 46.8 percent of high school students reported that they have 
ever had sexual intercourse, with 14.3 percent of students reporting that 
they had had sexual intercourse with four or more persons. The likelihood 
of ever having sexual intercourse varied by grade, with the highest rate 
among 12th grade students (63.1 percent) and the lowest rate among 9th 
grade students (34.3 percent). CDC also has reported that the prevalence 
of certain STDs — including the rate of chlamydia infection, the most 
frequently reported STD in the United States — peaks in adolescence and 
young adulthood. According to CDC, in 2004 the chlamydia rates among 
adolescents 15 to 19 years old (1,579 cases per 100,000 adolescents) and 
young adults 20 to 24 years old (1,660 cases per 100,000) were each more 
than twice the rates among all other age groups. 

HHS’s current strategic plan includes the objectives to reduce the 
incidence of STDs and unintended pregnancies and to promote family 
formation and healthy marriages. These two objectives support HHS’s 
goals to reduce the major threats to the health and well-being of 
Americans and to improve the stability and healthy development of 
American children and youth, respectively. Abstinence-until-marriage 
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education programs are one of several types of programs that support 
these objectives . 6 

The three main federal abstinence-until-marriage education programs — 
the State Program, the Community-Based Program, and the AFL 
Program — provide grants to support the recipients’ own efforts to provide 
abstinence-until-marriage education at the local level . 7 These programs 
must comply with the statutory definition of abstinence education (see 
table l ). 8 



6 HHS’s Family Planning Program, for example, also supports the objective to reduce the 
incidence of STDs and unintended pregnancies by providing access to contraceptive 
supplies and family planning information, especially for low-income persons, at community 
health clinics. This program is authorized under Title X of the Public Health Service Act. 

There are other federal sources of funding that are used for abstinence education, such as 
the Temporary Assistance for Needy Families (TANF) Program that is administered by 
ACF. Some states have allocated some of their TANF funding for abstinence education 
programs. For example, Florida has used TANF funds to provide community-based and 
faith-based organizations with contracts to carry out abstinence education. Other sources 
of funding that are used for abstinence education include ACF’s Compassion Capital Fund 
and CDC’s Division of Adolescent and School Health grants. 

8 42 U.S.C. § 710(b)(2). This definition is also referred to as the A-H definition. This statutory 
provision defines abstinence education for purposes of the State Program. Annual 
appropriations acts and program announcements have extended this definition to the 
Community-Based and AFL Programs. See, e.g , Departments of Labor, Health and Human 
Services, and Education, and Related Agencies Appropriations Act, 2006, Pub. L. No. 109- 
149,119 Stat. 2833, 2855-56. 
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Table 1: Definition of Abstinence Education 



The State Program 



Abstinence education refers to an educational or motivational program that: 

A. has as its exclusive purpose, teaching the social, psychological, and health gains 
to be realized by abstaining from sexual activity; 

B. teaches abstinence from sexual activity outside marriage as the expected standard 
for all school age children; 

C. teaches that abstinence from sexual activity is the only certain way to avoid out-of- 
wedlock pregnancy, sexually transmitted diseases, and other associated health 
problems; 

D. teaches that a mutually faithful monogamous relationship in context of marriage is 
the expected standard of human sexual activity; 

E. teaches that sexual activity outside of the context of marriage is likely to have 
harmful psychological and physical effects; 

F. teaches that bearing children out-of-wedlock is likely to have harmful 
consequences for the child, the child’s parents, and society; 

G. teaches young people how to reject sexual advances and how alcohol and drug 
use increases vulnerability to sexual advances; and 

H. teaches the importance of attaining self-sufficiency before engaging in sexual 
activity. 



Source: Social Security Act, § 510(b)(2) (codified at 42 U.S.C. § 710(b)(2)). 



The State Program, administered by ACF, provides funding to its 
grantees — states — for the provision of abstinence-until-marriage education 
to those most likely to have children outside of marriage. 9 States that 
receive grants through the State Program have discretion in how they use 
their funding to provide abstinence-until-marriage education. Some require 
that organizations apply for funds and use them to administer abstinence- 
until-marriage education programs. Others may directly administer such 
programs. At their discretion, states may also provide mentoring, 
counseling, and adult supervision to adolescents to promote abstinence 
from sexual activity until marriage. 

The Personal Responsibility and Work Opportunity Reconciliation Act of 
1996 established the State Program, and states were awarded grants 
beginning in fiscal year 1998. 10 Funds are allotted to each state that 
submits the required annual application based on the ratio of the number 



9 Through the State Program funds are also provided to insular areas and the District of 
Columbia. 

10 Pub. L. No. 104-193, § 912; 110 Stat. 2353-54 (codified at 42 U.S.C. § 710). 
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of low-income children in the state to the total number of low-income 
children in all states. States are required to match every $4 they receive in 
federal money with $3 of nonfederal money and are required to report 
annually on the performance of the abstinence-until-marriage education 
programs that they support or administer. In fiscal year 2005, 47 states, the 
District of Columbia, and 3 insular areas were awarded funding. 11,12 



The Community-Based 
Program 



The Community-Based Program, which is also administered by ACF, is 
focused on funding public and private entities that provide abstinence- 
until-marriage education for adolescents from 12 to 18 years old, with the 
purpose of creating an environment within communities that supports 
adolescent decisions to postpone sexual activity until marriage. The 
Community-Based Program provides grants for school-based programs, 
adult and peer mentoring, and parent education groups. The Community- 
Based Program first awarded grants in fiscal year 2001. 13 Grantees of the 
Community-Based Program are selected through a competitive process 
and are evaluated according to several criteria, such as the extent to which 
they have demonstrated that a need exists for abstinence-until-marriage 
education for a targeted population or in a specific geographic location. 
Grantees are required to report to ACF, on a semiannual basis, on the 
performance of their programs. For fiscal year 2005, 63 grants were 
awarded to organizations and other entities. 14 



u In this report, we refer to U.S. territories and commonwealths as “insular areas.” 

12 Some states and insular areas have not applied for funding under the State Program. 
California, Maine, and Pennsylvania did not apply for funding under the State Program in 
fiscal year 2005. In this report, when we refer to “states,” we are referring to all grantees of 
the State Program — including states, insular areas, and the District of Columbia. 

13 The Community-Based Program is conducted under section 1110 of the Social Security 
Act. See 42 U.S.C. § 1310. 

14 In addition to the 63 grants awarded in fiscal year 2005, ACF is also responsible for other 
grants that the agency awarded before 2005. 
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The AFL Program The AFL Program supports programs that provide abstinence-until- 

marriage education. 15 The primary purpose of these programs is to find 
effective means of reaching preadolescents and adolescents before they 
become sexually active in order to encourage them to abstain from sexual 
activity and other risky behaviors. Under the AFL Program, OPA awards 
competitive grants to public or private nonprofit organizations or 
agencies, including community-based and faith-based organizations, to 
facilitate abstinence-until-marriage education in a variety of settings, 
including schools and community centers. Established in 1981, the AFL 
Program began awarding grants in fiscal year 1982. AFL Program grantees 
include school districts, youth development groups, and medical centers. 
Grant applicants are evaluated based on several criteria, such as the extent 
to which they provide a clear statement of mission, goals, measurable 
objectives, and a reasonable method for achieving their objectives. 
Grantees are required to conduct evaluations of certain aspects of their 
programs and report annually on their performance. As of August 2006, 
OPA funded 58 abstinence-until-marriage education programs, and most of 
these were focused on reaching young adolescents from the ages of 9 to 
14. 



Funding for Abstinence- 
until-Marriage Education 
Has Increased 



Funding provided by HHS for abstinence-until-marriage education 
programs has increased steadily since 2001 (see table 2). For the three 
main programs combined — the State Program, the Community-Based 
Program, and the AFL Program — the amount of agency funding increased 
from about $73 million in fiscal year 2001 to about $158 million in fiscal 
year 2005. Nearly all of this increase was for the Community-Based 
program; funding under this program increased by about $84 million from 
fiscal years 2001 through 2005. In fiscal year 2005, agency funding for the 
Community-Based Program constituted the largest share of the total 
funding (about 66 percent) for the three main programs combined. 



15 See 42 U.S.C. § 300z et seq. The AFL Program also supports other projects for pregnant 
and parenting adolescents, their infants, male partners, and family members. The purpose 
of these projects is to improve the outcomes of early childbearing for teen parents, their 
infants, and their families. However, in this report, when we use the term “AFL Program,” 
we are referring only to the abstinence-until-marriage component of the AFL Program. 
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Table 2: Funding Provided by HHS for the Three Main Abstinence-until-Marriage 
Education Programs 



Dollars in millions 



Fiscal year 


State Program 86 


Community-Based Program 


AFL Program 


2001 


$43 


$20 


$10 


2002 


43 


40 


12 


2003 


44 


55 


13 


2004 


41 


75 


13 


2005 


41 


104 


13 


Sources: ACF, OPA, and HRSA. 



Notes: Figures are rounded to nearest $1,000,000. Funding levels represent the total amount of 
grants awarded and funding for program-related support, such as technical assistance and 
evaluation. 

“States that receive funding are required to match every $4 they receive of federal funds with $3 of 
nonfederal money. 

b The amount of funding provided by HHS for the State Program has generally varied by year because 
the states that have applied for funding each year have varied. 

Within each of the three main abstinence-until-marriage education 
programs, the amount of individual grants varied. 16 In fiscal year 2005, the 
State Program’s annual grants ranged from $57,057 to $4,777,916 and the 
median annual grant amount was $569,675. That same year, the 
Community-Based Program’s annual grants ranged from $213,276 to 
$800,000 and the median grant amount was $642,250. In fiscal year 2006, 
the AFL Program’s annual grants ranged from $95,676 to $300,000 and the 
median grant amount was $225,000. 



Federal Agency 
Responsibilities Related to 
Abstinence-until-Marriage 
Education 



Five organizational units located within HHS — ACF, OPA, CDC, ASPE, and 
NIH — have responsibilities related to abstinence-until-marriage education. 
ACF and OPA administer the three main federal abstinence-until-marriage 
education programs. CDC supports abstinence-until-marriage education at 
the national, state, and local levels. CDC, ASPE, and NIH are sponsoring 
research on the effectiveness of abstinence-until-marriage programs. 



16 ACF awards formula grants under the State Program each year, and states have 2 years to 
spend the funds they are awarded. In the Community-Based Program and AFL Program, 
grantees develop multiyear projects — up to 5 years — for which the first year of funding is 
provided through competitive grants; for subsequent years, grantees may obtain funding 
through noncompetitive continuation grants. 
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ACF ACF is responsible for federal programs that promote the economic and 

social well-being of families, children, individuals, and communities. ACF 
administers and provides oversight of both the State Program and the 
Community-Based Program by, among other things, awarding grants, 
providing training and technical assistance to grantees, and requiring 
annual performance reporting from grantees. ACF has been responsible 
for the State Program since June 2004 and the Community-Based Program 
since October 2005. HRSA previously administered these programs. 

OPA OPA has responsibility for advising the Secretary of HHS on a wide range 

of reproductive health topics, including adolescent pregnancy and family 
planning. The office is also responsible for administering programs that 
provide services for pregnant and parenting teens and prevention 
programs, such as abstinence-until-marriage education programs. OPA 
administers and provides oversight of the AFL Program by awarding 
grants, providing training and technical assistance to grantees, and 
requiring annual performance reporting from grantees. 

CDC CDC is primarily responsible for the prevention and control of infectious 

and chronic diseases, including STDs. CDC provides funding to state and 
local education agencies in their efforts to support comprehensive school 
health education and HIV/STD prevention education programs, and CDC 
officials told us that some of these are focused on abstinence. CDC also 
provides funding to several state education agencies to implement various 
abstinence projects, such as collaboration-building among agencies to 
increase the impact of their efforts to encourage abstinence. Further, CDC 
develops tools to assist state and local education agencies with their 
health education programs. CDC provides funding to several national 
organizations to build the capacity of abstinence-until-marriage education 
providers. Organizations’ activities include, but are not limited to, the 
development and distribution of educational materials. CDC is also 
sponsoring research on the effectiveness of an abstinence-until-marriage 
education program. 

ASPE ASPE advises the Secretary of HHS in several areas, including policy 

development in health, human services, data, and science. ASPE is 
responsible for the development of policy analyses and it conducts 
research and evaluation studies in several areas, including the health of 
children and adolescents. ASPE is currently sponsoring research on the 
effectiveness of abstinence-until-marriage education programs. 

NIH NIH is the primary federal agency that conducts and supports medical and 

behavioral research among various populations, including children and 
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adolescents. NIH is currently sponsoring research on the effectiveness of 
abstinence-until-marriage education programs. 



Federal and State 
Efforts to Assess the 
Scientific Accuracy of 
Materials Used in 
Abstinence-until- 
Marriage Education 
Programs Have Been 
Limited 



Efforts by HHS and states to assess the scientific accuracy of materials 
used in abstinence-until-marriage education programs have been limited. 
ACF — which awards grants to two programs that account for the largest 
portion of federal spending on abstinence-until-marriage education — does 
not review its grantees’ education materials for scientific accuracy and 
does not require grantees of either program to review their own materials 
for scientific accuracy. In addition, not all states funded through the State 
Program have chosen to review their program materials for scientific 
accuracy. In contrast to ACF, OPA has reviewed the scientific accuracy of 
grantees’ proposed educational materials and corrected inaccuracies in 
these materials. 



ACF Does Not Review 
Program Materials for 
Scientific Accuracy and 
Does Not Require Grantees 
to Do So, though Some 
State Grantees Have 
Conducted Such Reviews 



There have been limited efforts to review the scientific accuracy of 
educational materials used in ACF’s State and Community-Based 
Programs — the two programs that account for the largest portion of 
federal spending on abstinence education. ACF does not review materials 
for scientific accuracy in either reviewing grant applications 17 or in 
overseeing grantees’ performance. Prior to fiscal year 2006, State Program 
and Community-Based Program applicants were not required to submit 
copies of their proposed educational materials with their applications. 
While ACF required grantees of the Community-Based Program — but not 
the State Program — to submit their educational materials with their fiscal 
year 2006 applications, ACF officials told us that grantee applications and 
materials are only reviewed to ensure that they address all aspects of the 
scope of the Community-Based Program, such as the A-H definition of 
abstinence education. 18 Further, documents provided to us by ACF 
indicate that the agency does not review grantees’ educational materials 
for scientific accuracy as a routine part of its oversight activities. In 
addition, ACF also does not require its grantees to review their own 



17 In reviewing grantees applications, ACF does examine several issues, including 
applicants’ stated program goals and need for assistance, their compliance with the A-H 
definition of abstinence education, their intended approach in carrying out their objectives, 
and their budget plan. 

18 HHS officials told us that if ACF finds inaccurate statements during this more general 
review process or if inaccuracies are brought to their attention at any time during the grant 
period, ACF officials work with the grantees to take corrective action. 
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materials for scientific accuracy. Similarly, when HRSA was responsible 
for the State and Community-Based Programs, the agency did not review 
materials used by grantees for scientific accuracy or require grantees to 
review their own materials. 

Not all grantees of the State Program have chosen to review the scientific 
accuracy of their educational materials. Officials from 5 of the 10 states in 
our review reported that their states have chosen to conduct such 
reviews. 19,20 Officials in these states identified a variety of reasons why 
their states reviewed abstinence-until-marriage educational materials, 
including program requirements, state education laws and guidelines, and 
past lawsuits, to ensure that materials used in abstinence-until-marriage 
programs were accurate. For example, Michigan’s Revised School Code 
states that materials and instruction in the sex education curricula, 
including information on abstinence, “shall not be medically inaccurate,” 21 
and Ohio’s fiscal year 2007 abstinence-until-marriage education program 
guidance states that abstinence-until-marriage educational materials 
“should be medically accurate in all assertions.” 22 

The five states we contacted that review abstinence-until-marriage 
educational materials for scientific accuracy have used a variety of 
approaches in their reviews. Some states contracted with medical 
professionals — such as nurses, gynecologists, and pediatricians — to serve 
as medical advisors who review program materials and use their expertise 
to determine what is and is not scientifically accurate. Some states have 
created checklists or worksheets to guide their staff conducting the review 
and document findings of inaccuracy or verification of a statement. All five 
states use medical professionals in conducting these reviews. One of the 
states requires that all statistics or scientific statements cited in a 



19 In addition to reviewing materials for accuracy, one state requires abstinence-until- 
marriage providers to sign a written assurance that their materials are scientifically 
accurate. Officials from this state also reported providing abstinence-until-marriage 
education programs with public health consultants to provide technical assistance and 
training to help ensure the accuracy of their educational materials. 

20 In addition, some state officials we interviewed told us that review committees for local 
school districts may review the scientific accuracy of educational materials that include 
information about HIV and other STDs, including abstinence-until-marriage education 
materials. 

21 Mich. Comp. Laws Ann. § 380. 1507b(2)(West 2004). 

22 See Ohio Department of Health, “Abstinence Education Program Request for Proposals 
for Fiscal Year 2007,” (program announcement, 2005). 
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program’s materials are sourced to CDC or a peer-reviewed medical 
journal. Officials from this state told us that if statements in these 
materials cannot be attributed to these sources, the statements are 
required to be removed until citations are provided and materials are 
approved. Officials from this state told us they have also supplemented 
their review of program materials with on-site classroom observations to 
assess the scientific accuracy of the information presented to students. 

Officials from two of the five states reported that they have found 
inaccuracies as a result of their reviews. For example, one state official 
stated that because information is constantly evolving, state officials have 
had to correct out-of-date scientific information. In addition, this official 
cited an instance where materials incorrectly suggested that HIV can pass 
through condoms because the latex used in condoms is porous. In 
addition, this official provided documentation that the state has had to 
correct a statement indicating that when a person is infected with the 
human papillomavirus, 23 the virus is “present for life” because, in almost all 
cases, this is not true. State officials who have identified inaccuracies told 
us that they informed their grantees of inaccuracies so that they could 
make corrections in their individual programs. One state official added 
that she contacted the authors of the materials to report an inaccuracy. 

Some of the educational materials that states have reviewed are materials 
that are commonly used in the Community-Based Program. Officials from 
four of the five states that review materials for scientific accuracy told us 
that they have each reviewed at least one of the five curricula most 
commonly used in the Community-Based Program because programs in 
their state were using them: Choosing the Best, WAIT Training, Sex Can 
Wait, A.C. Green’s Game Plan Abstinence Program, and Worth the Wait. 
Based on ACF documents, we found that there were 58 different curricula 
used by grantees of the Community-Based Program in fiscal year 2005. 
However, more than half of the grantees of the Community-Based Program 
reported using at least one of these five curricula. 24 

While there has been limited review of materials used in the State and 
Community-Based Programs, grantees of these programs have received 
some technical assistance designed to improve the scientific accuracy of 



23 The human papillomavirus (HPV) causes an STD called genital HPV infection. 

24 Some grantees of the Community-Based Program reported using more than one of these 
curricula in fiscal year 2005. 
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their materials. For example, ACF officials reported that the agency 
provided a conference for grantees of the Community-Based Program in 
February 2006 that included a presentation focused on medical accuracy, 
including a discussion of state legislative proposals that would require 
medical accuracy in abstinence-until-marriage education, and how to 
identify reliable data. In addition, in 2002, HRSA awarded a contract to the 
National Abstinence Clearinghouse requiring, among other things, that the 
contractor develop and implement a program to provide medically 
accurate information and training to grantees of the State and Community- 
Based Programs. 25 (See app. I for a description of HRSA’s process for 
awarding this contract). The portion of the contract that focused on 
providing medically accurate information to grantees was subcontracted 
to the Medical Institute for Sexual Health (Medical Institute), 26 which has 
conducted presentations at regional educational conferences to provide 
grantees with medical and scientific information, such as updated 
information on condoms and STD transmission. The Medical Institute has 
also provided consultative services to grantees by responding to medical 
and scientific questions. 



OPA Reviews Materials 
Used by AFL Program 
Grantees for Scientific 
Accuracy 



In contrast to ACF, OPA reviews for scientific accuracy the educational 
materials used by AFL Program grantees. Specifically, OPA reviews its 
grantees’ proposed educational materials for scientific accuracy before 
they are used. Agency officials stated that they began to review these 
materials while litigation concerning the AFL Program was ongoing. OPA 
continued to review these materials as part of a 1993 settlement to this 
lawsuit. 27 The settlement agreement expired in 1998, though the agency has 
continued to review grantees’ proposed educational materials for accuracy 
as a matter of policy. OPA officials told us that grant applicants submit 
summaries of materials they propose to use, though the materials are not 
reviewed for scientific accuracy until after grantees have been selected. 
OPA officials said that after grants are awarded, a medical education 
specialist (in consultation with several part-time medical experts) reviews 
the grantees’ printed materials and other educational media, such as 



25 The administration of this contract was transferred to ACF in May 2005. 

26 The Medical Institute is a nonprofit organization that provides educational resources, 
conferences, and seminars to educators, health professionals, pregnancy care centers, and 
faith-based groups about behaviors to decrease STDs and out-of-wedlock pregnancies, 
including abstinence. 

27 See generally Bowen v. Kendrick, 487 U.S. 589 (1988), 657 F. Supp 1547 (D.D.C. 1987). 
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videos. OPA officials explained that the medical education specialist must 
approve all materials before they are used. On many occasions, OPA 
grantees have proposed using — and therefore OPA has reviewed — 
materials commonly used in the Community-Based Program. For example, 
an OPA official told us that the agency had reviewed three of the 
Community-Based Program’s commonly used curricula — Choosing the 
Best, Sex Can Wait, and A. C. Green’s Game Plan Abstinence Program — 
and is also currently reviewing another curriculum commonly used by 
Community-Based Program grantees, WAIT Training. 29 

OPA officials stated that the medical education specialist has occasionally 
found and addressed inaccuracies in grantees’ proposed educational 
materials. OPA officials stated that these inaccuracies are often the result 
of information being out of date because, for example, medical and 
statistical information on STDs changes frequently. OPA has addressed 
these inaccuracies by either not approving the materials in which they 
appeared or correcting the materials through discussions with the 
grantees and, in some cases, the authors of the materials. In fiscal year 
2005, OPA disapproved of a grantee using a specific pamphlet about STDs 
because the pamphlet contained statements about STD prevention and 
HIV transmission that were considered incomplete or inaccurate. For 
example, the pamphlet stated that there was no cure for hepatitis B, but 
the medical education specialist required the grantee to add that there was 
a preventive vaccine for hepatitis B. In addition, OPA required that a 
grantee correct several statements in a true/false quiz — including 
statements about STDs and condom use — in order for the quiz to be 
approved for use. For example, the medical education specialist changed a 
sentence from “The only 100% effective way of avoiding STDs or unwanted 
pregnancies is to not have sexual intercourse.” to “The only 100% effective 
way of avoiding STDs or unwanted pregnancies is to not have sexual 
intercourse and engage in other risky behaviors.” 



An addition, a CDC official told us that some of its grantees are producing educational 
materials with CDC funds to be used by abstinence-until-marriage education programs, 
which are likely to include State and Community-Based Program grantees. These materials 
are required to be reviewed for scientific accuracy. CDC. officials told us that they have 
made corrections to some of these materials. Materials used in school-based HIV 
prevention education programs that are supported with CDC funds are also reviewed for 
scientific accuracy. A CDC official told us that some of these programs are abstinence- 
until-marriage education programs. 
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While OPA and some states have reviewed their grantees’ abstinence-until- 
marriage education materials for scientific accuracy , 29 these types of 
reviews have the potential to affect abstinence-until-marriage education 
providers more broadly. Such efforts may create an incentive for authors 
of abstinence-until-marriage education materials to ensure they are 
accurate. Thus, some authors of abstinence-until-marriage education 
materials have recently updated materials in their curricula following 
reports that questioned their accuracy. For example, one of the most 
widely used curricula used by grantees of the Community-Based 
Program — WAIT Training — has been recently updated and provides the 
updated information on its Web site. A representative from WAIT Training 
stated that the company recently revised its curriculum, in part, in 
response to a congressional review that found inaccuracies in its 
abstinence-until-marriage education materials. 



A Variety of Efforts 
Have Been Made to 
Assess the 
Effectiveness of 
Abstinence-until- 
Marriage Education 
Programs, but a 
Number of Factors 
Limit the Conclusions 
That Can Be Drawn 



HHS, states, and researchers have made a variety of efforts to assess the 
effectiveness of abstinence-until-marriage education programs; however, a 
number of factors limit the conclusions that can be drawn about the 
effectiveness of these programs. ACF and OPA have required their 
grantees to report on various outcomes used to measure the effectiveness 
of grantees’ abstinence-until-marriage education programs , 30 though the 
reporting requirements for each of the three abstinence-until-marriage 
programs differ. In addition, to assess the effectiveness of the State and 
Community-Based Programs, ACF has analyzed national data on 
adolescent birth rates and the proportion of adolescents who report 
having had sexual intercourse. Other organizational units within HHS — 
ASPE, CDC, and NIH — are funding studies designed to assess the 
effectiveness of abstinence-until-marriage education programs in delaying 
sexual initiation, reducing pregnancy and STD rates, and reducing the 
frequency of sexual activity. Despite these efforts, several factors limit the 



29 In addition to OPA and some states, others have also reviewed the scientific accuracy of 
abstinence-until-marriage education materials. See, for example, Wilson et al. 

30 This reporting is a part of ACF’s efforts to collect evaluative information about these 
programs. These efforts include both performance measurement — the ongoing monitoring 
and reporting of program accomplishments toward preestablished goals — and program 
evaluation — individual systematic studies to assess how well a program is working. Both 
types of assessments aim to support decisions to improve service delivery and program 
effectiveness. See GAO, Performance Measurement and Evaluation: Definitions and 
Relationships, GAO-05-739SP (Washington, D.C.: May 2005), for more information on types 
of assessments. 
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conclusions that can be drawn about the effectiveness of abstinence-until- 
marriage education programs. Most of the efforts to evaluate the 
effectiveness of abstinence-until-marriage education programs that we 
describe in our review have not met certain minimum criteria that experts 
have concluded are necessary in order for assessments of program 
effectiveness to be scientifically valid, in part because such designs can be 
expensive and time-consuming to carry out. In addition, the results of 
some efforts that meet the criteria of a scientifically valid assessment have 
varied, and two key studies that meet these criteria have not yet been 
completed. 



HHS, States, and 
Researchers Have Made a 
Variety of Efforts to Assess 
the Effectiveness of 
Abstinence-until-Marriage 
Education Programs 



Efforts of HHS, states, and researchers to assess the effectiveness of 
abstinence-until-marriage education programs have included ACF and 
OPA requiring grantees to report data on outcomes of their abstinence- 
until-marriage education programs; ACF analyzing national data on 
adolescent behavior and birth rates; and other HHS agencies, states, and 
researchers funding or conducting studies to assess the effectiveness of 
abstinence-until-marriage education programs. 



ACF and OPA Have Required ACF has made efforts to assess the effectiveness of abstinence-until- 
Grantee Reporting of Data on marriage education programs funded by the State Program and the 
Outcomes Community-Based Program. One of ACF’s efforts has been to require 

grantees of both programs to report data on outcomes, though the two 
programs have different requirements for the outcomes grantees must 
report. For the State Program, as of fiscal year 2006, grantees must report 
annually on four measures of the prevalence of adolescent sexual behavior 
in their states, such as the rate of pregnancy among adolescents aged 16 to 
17 years, and compare these data to program targets over 6 years. To 
report on these four measures, states may choose the data sources they 
will use. 31 States must also develop and report on two additional 
performance measures that are related to the goals of their programs. 32 



31 Previously, to report on the four measures, states have relied on either state or national 
data sources, such as CDC’s Youth Risk Behavior Surveillance System. 

32 For example, in fiscal year 2002, state grantees developed such measures as the 
percentage of teens surveyed who show an increase in participating in structured activities 
after school hours; the percentage of live births to women younger than 18, fathered by 
men age 20 and older; the percentage of program participants proficient in refusal skills; 
the percentage of high school students who reported using drugs or alcohol before 
intercourse; and the percentage of high school students who had sexual intercourse for the 
first time before age 13. 
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(See table 3 for a list of ACF’s fiscal year 2006 reporting requirements for 
the State Program.) 



Table 3: ACF’s Reporting Requirements for the State Program and the Community- 
Based Program, Fiscal Year 2006 



State Program 

• Report on four performance measures: 
(1) rate of pregnancy among female 
teenagers aged 15 to 17, (2) proportion 
of adolescents who have engaged in 
sexual intercourse, (3) incidence of 
youths 15 to 19 years old who have 
contracted selected STDs, and (4) rate 
of births among female teenagers aged 
15 to 17. 

• Develop and report on two additional 
performance measures that are related 
to individual program goals. Past 
examples of these additional measures 
have included the percentage of high 
school students who reported using 
drugs or alcohol before intercourse and 
the percentage of high school students 
who had sexual intercourse for the first 
time before age 13. 



Community-Based Program 

• Report on program goals that are 
developed by grantees with a third-party 
evaluator. Such outcomes could include 
changes in knowledge about abstinence 
or declared behavior among participants 
of abstinence-until-marriage programs. 



• Report on program “outputs”: the number 
of youth served, the hours of service 
provided to each youth, and the number 
of youths who complete the program. 
Grantees choose additional outputs that 
allow for effective monitoring and 
management of the project. The 
additional outputs may include tracking 
the number of staff trained to provide 
services, the number of events hosted, 
number of marketing materials 
distributed, and so forth. 



Sources: State and Community-Based Programs’ announcements, fiscal year 2006. 



As of fiscal year 2006, ACF requires Community-Based Program grantees 
to develop and report on outcome measures designed to demonstrate the 
extent to which grantees’ community-based abstinence education 
programs are accomplishing their program goals. 33 ACF requires grantees 
of the Community-Based Program to contract with third-party evaluators, 
who are responsible for both helping grantees develop the outcome 
measures and monitoring grantee performance against the measures, 34 but 



33 The fiscal year 2006 program announcement for the Community-Based Program provides 
examples of outcome measures that grantees could use, including increased knowledge of 
the benefits of abstinence, the number of youths who commit to abstaining from premarital 
sexual activity, and increased knowledge of how to avoid high-risk situations and risk 
behaviors. 

34 Fiscal year 2006 Community-Based Program grantees are required to devote a minimum 
of 15 percent of their requested budgets to performance monitoring by third-party 
contractors. 
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because this is a new requirement established for fiscal year 2006 grantees, 
ACF has not yet received the results of these evaluations. In addition to 
outcome reporting, ACF requires grantees of the Community-Based 
Program to report on program “outputs,” which measure the quantity of 
program activities and other deliverables, such as the number of 
participants who are served by the abstinence-until-marriage education 
programs. According to ACF officials, the agency requires grantees of both 
the State Program and the Community-Based Program to report on 
program outcomes in order to monitor grantees’ performance, target 
training, and technical assistance, and help grantees improve service 
delivery. (See table 3 for a list of ACF’s fiscal year 2006 reporting 
requirements for the Community-Based Program.) 

ACF’s fiscal year 2006 reporting requirements for grantees of the State 
Program are the same as HRSA’s when it administered the State Program. 
In contrast, ACF’s fiscal year 2006 reporting requirements for the 
Community-Based Program differ from HRSA’s reporting requirements for 
the program. For example, for Community-Based Program grants awarded 
in fiscal year 2001, HRSA required 35 grantees to report on the effectiveness 
of their programs, as measured by program participation as well as 
behavioral and biological outcomes. 36 These performance measures were 
modified for fiscal year 2002, in part HHS officials explained, because of 
concerns expressed by members of the abstinence-education community 
that the original performance measures did not accurately reflect the 
efforts of the grantees of the Community-Based Program. For grants 
awarded from fiscal years 2002 through 2004, HRSA required grantees of 
the Community-Based Program to report on a combination of program 
outputs, such as the proportion of adolescents who completed an 
abstinence-until-marriage education program, and measures of adolescent 
intentions, such as the proportion of adolescents who committed to 



35 Some grantees of the Community-Based Program may have to meet reporting 
requirements established by HRSA. Grants under this program are awarded for projects 
that may extend over a period of several years. Grantees that were awarded grants when 
HRSA administered the program and have since received noncompetitive continuation 
grants for these projects are required to meet the reporting requirements in place at the 
time they first received the competitively awarded grants. 

3tJ In fiscal year 2001, HRSA required grantees of the Community-Based Program to report 
on the following four performance measures: the proportion of program participants who 
successfully complete or remain enrolled in an abstinence-only education program; the 
proportion of program participants who have engaged in sexual intercourse; the proportion 
of program participants who report a reduction in risk behaviors, such as tobacco, alcohol, 
and drug use; and the rate of births to female program participants. 
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abstaining from sexual activity until marriage. 37 For grants awarded in 
fiscal year 2005, when ACF assumed responsibility for the Community- 
Based Program from HRSA, grantees were not required to report on any 
specific performance measures. 

OPA has also made efforts to assess the effectiveness of the AFL Program. 
Specifically, OPA requires grantees of the AFL Program to develop and 
report on outcome measures that are used to help demonstrate the extent 
to which grantees’ programs are having an effect on program 
participants. 38,39 According to OPA officials, the agency recommends that 
grantees report on outcome measures, such as participants’ knowledge of 
the benefits of abstinence and their reported intentions to abstain from 
sexually activity, reported beliefs in their ability to remain abstinent, and 
reported parental involvement in their lives. To collect data on these 
outcome measures and any others, OPA requires all grantees funded in 
fiscal year 2004 and beyond to administer, at a minimum, a standardized 
questionnaire — developed by OPA — to their program participants, both 
when participants begin an abstinence-only education program and after 



3 Specifically, HRSA required grantees of the Community-Based Program to report annually 
on the following six performance measures: the proportion of program participants who 
successfully completed or remained enrolled in an abstinence-only education program; the 
proportion of adolescents who understood that abstinence from sexual activity is the only 
certain way to avoid out-of-wedlock pregnancy and STDs; the proportion of adolescents 
who indicated an understanding of the social, psychological, and health gains to be realized 
by abstaining from premarital sexual activity; the proportion of participants who reported 
that they had the skills necessary to resist sexual urges and advances; the proportion of 
youth who committed to abstaining from sexual activity until marriage; and the proportion 
of participants who intended to avoid situations and risk behaviors, such as drug use and 
alcohol consumption, which make them more vulnerable to sexual advances and urges. 

38 In addition to these outcomes, grantees of the AFL Program are required to report on 
program outputs, such as the number of program participants, the average number of 
participants per session, and the average number of sessions attended by participants. 
Agency officials stated that OPA has implemented a new format for its grantees’ reports, 
which is intended to standardize their reporting on these outputs. 

39 OPA’s grantees are required to perform evaluations of their programs that are directly tied 
to their program objectives. For these evaluations, OPA requires grantees to develop 
research hypotheses that reflect the outcomes the grantees intend to achieve. This type of 
evaluation is generally considered to be an outcome evaluation — which assesses the extent 
to which a program achieves its outcome-oriented objectives. These evaluations focus on 
outputs and outcomes to judge program effectiveness but may also assess program process 
to understand how outcomes are produced. In addition, grantees of the AFL Program are 
required to perform implementation evaluations. 



Page 23 



GAO-07-87 Abstinence Education 




the program’s completion. 40 The standardized questionnaire includes 
questions intended to obtain information on participants’ reported 
involvement in extracurricular activities, behaviors linked to health risks, 
attitudes and intentions about abstinence, and opinions about the 
consequences of premarital sexual activity. Like ACF, OPA requires its 
grantees to contract with independent evaluators, such as colleges or 
universities, which are responsible for evaluating the effectiveness of 
grantees’ individual abstinence-until-marriage education programs. 41 In 
addition to evaluating the extent to which grantees are meeting their goals, 
OPA officials stated that the independent evaluators may also provide 
input to grantees of the AFL Program on other aspects of the program to 
improve their service delivery. Unlike ACF, OPA requires that the third- 
party evaluations incorporate specific methodological characteristics, 
such as control groups or comparison groups 42 and sufficient sample 
sizes. 43 In addition, OPA requires that the evaluations for grantees funded 
in fiscal year 2004 and beyond account for baseline and follow-up data 
obtained from the standardized questionnaires. 

OPA’s requirement that grantees use a standardized set of questionnaires, 
with data from these questionnaires used in evaluations, differs from 
OPA’s previous requirements. Previously, grantees of the AFL Program 
were not required to use a standardized method for collecting data that 



40 OPA officials stated that grantees may also supplement the standardized questionnaire 
with additional data collection instruments to obtain information on the effectiveness of 
their abstinence-until-marriage education programs. 

41 OPA has required that its grantees perform independent evaluations of their programs 
since the program first awarded grants in 1982, and requires that grantees devote from 
1 percent to 5 percent of grant funds to the evaluation of their programs. In cases where a 
more rigorous or comprehensive evaluation is proposed, OPA may allow these grantees to 
use up to 25 percent of their grant funds. 

42 A control group is a group of individuals or communities in a study that is compared to an 
intervention group — a group in a study that is receiving or participating in the program 
being studied. A control group is a randomly assigned group that does not receive the 
program. A comparison group is not randomly assigned like a control group. However, 
individuals or communities in well-matched comparison groups should have similar 
characteristics. 

43 Specifically, OPA requires that third-party evaluations of grantees of the AFL Program 
compare, when possible, randomized control or matched comparison groups with groups 
receiving abstinence-until-marriage education. In addition, OPA requires that these 
evaluations include a sufficient sample size to ensure that any observed differences 
between the groups are statistically valid and that the evaluations include a follow-up 
assessment of program participants at least 6 months after the abstinence-until-marriage 
intervention has been tested. 
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could be used to assess the effectiveness of their programs; instead, 
grantees chose their own data collection instruments. As a result, an OPA 
official explained, the collected data varied from one project to another. 
OPA officials said that the agency developed the standardized 
questionnaire to ensure uniformity in the data collected and allow the 
agency to more effectively aggregate the data reported in evaluations of 
individual abstinence-until-marriage education programs. 

OPA officials told us that they plan to aggregate information from certain 
questions in the standardized set of questionnaires in order to report on 
certain performance measures as part of the agency’s annual performance 
reports. 44 The measures include the extent of parental involvement in 
adolescents’ lives and the extent to which adolescents understand the 
benefits of abstinence. An agency official stated that the agency expects to 
begin receiving data from grantees that are using these questionnaires in 
January 2007. OPA did not previously have long-term measures of the 
performance of the AFL Program. Its current measures were developed in 
collaboration with the Office of Management and Budget (OMB) in 
response to an OMB review in 2004 that found that the AFL Program did 
not have any annual performance measures for measuring progress 
toward long-term goals. 

In addition to requiring their grantees to report on outcomes used to 
assess program effectiveness, both ACF and OPA have provided technical 
assistance and training to their grantees in order to support grantees’ own 
program evaluation efforts. For example, in November 2005 the two 
agencies sponsored an evaluation conference for abstinence-until- 
marriage grantees that included presentations about evaluations and their 
methodology. Similarly, ACF’s Office of Planning, Research, and 
Evaluation sponsors annual evaluation conferences, and an ACF official 
told us that a recent conference placed “a significant emphasis” on the 
evaluation of abstinence-until-marriage education programs. In addition, 
HHS officials told us that ACF, along with ASPE, is funding a multiyear 
project that is designed to identify gaps in abstinence education evaluation 
and technical assistance needs, develop materials on abstinence education 
evaluation, deliver technical assistance and capacity-building activities 
related to program evaluation, and develop research reports related to 



44 OPA prepares annual performance reports as a part of HHS’s responsibilities under the 
Government Performance and Results Act, which include program performance measures 
to help link funding decisions with performance and review of related outcome measures. 



Page 25 



GAO-07-87 Abstinence Education 





ACF Uses National Data on 
Adolescent Behavior as a 
Measure of the Overall 
Effectiveness of Its Two 
Programs 



abstinence education. OPA officials also told us that they attempt to help 
ensure grantees’ progress and effectiveness by offering various technical 
assistance workshops and conferences. For example, in May 2006 OPA 
provided a 2-day training conference to its grantees on the importance of 
program evaluations and administering evaluation instruments. In 
addition, OPA officials stated that the agency contracts with evaluation 
consultants, who review grantees’ evaluation tools and activities. OPA 
officials explained that these consultants provide in-depth technical 
assistance to grantees on how to improve grantees’ evaluations. 

Requiring outcome reporting from state and community-based grantees is 
not ACF’s only effort to assess the effectiveness of its two programs. ACF 
also analyzes trends in adolescent behavior, as reflected in national data 
on birth rates among teens and the proportion of surveyed high school 
students reporting that they have had sexual intercourse. 46 ACF uses these 
national data as a measure of the overall effectiveness of its State and 
Community-Based Programs, comparing the national data to program 
targets. In its annual performance reports, the agency summarizes the 
progress being made toward lowering the rate of births to unmarried 
teenage girls and the proportion of students (grades 9-12) who report 
having ever had sexual intercourse. 

ACF’s use of national data to assess the effectiveness of the State and 
Community-Based Programs represents a change from how HRSA 
assessed the overall effectiveness of these programs. Whereas ACF 
compares national data on adolescent behavior to program targets, HRSA 
aggregated data from its state and community-based grantees. HRSA’s 
state grantees were allowed to select the data sources used to gauge their 
progress against certain performance measures. For example, in its annual 
performance reports on the State Program, HRSA reported information on 
the percentage of its state grantees meeting target rates for reducing the 
proportion of adolescents who have engaged in sexual intercourse, the 
incidence of youths aged 15 to 19 who have contracted selected STDs, and 
the rate of births among youths aged 15 to 17. To determine their progress 
in meeting their target rates, some state grantees, for example, reported 
national data from the Youth Risk Behavior Surveillance System, while 



46 Data on teen birth rates and adolescents’ reported sexual behavior are contained in the 
National Vital Statistics System and the Youth Risk Behavior Surveillance System, 
respectively. The former is a national data set of public health statistics reported by states 
to CDC, and the latter is a national data set based on nationwide surveys administered to 
high school students by CDC. 
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Some States Have Made 
Additional Efforts to Assess the 
Effectiveness of Abstinence- 
until-Marriage Education 
Programs 



other grantees reported state-collected data. After ACF assumed 
responsibility for the State and Community-Based Programs from HRSA, 
ACF began using national data on adolescent behavior as a measure of the 
programs’ effectiveness. According to ACF officials, the agency changed 
how it assessed its programs out of concern over the quality of the data 
state grantees were using in their performance reporting and because the 
agency wanted to use parallel measures of effectiveness for both 
programs. For example, according to state performance reports for fiscal 
year 2001 that we reviewed, two reports did not include adolescent 
pregnancy rates that year because the states did not collect data on 
abortions among this population. 46 In addition, ACF officials told us that 
they decided not to use national data on STDs as a measure of program 
effectiveness because the goal of reducing STD rates is not as central to 
the State and Community-Based Programs as reducing sexual activity and 
birth rates among teens. However, one official stated that reducing STDs is 
an important “by-product” of the programs. 

Some states have made additional efforts to assess the effectiveness of 
abstinence-until-marriage education programs, although they are not 
required by ACF. 47 Specifically, we found that 6 of the 10 states in our 
review that receive funding through ACF’s State Program have made 
efforts to conduct evaluations of selected abstinence-until-marriage 
programs in their state. All 6 of the states worked with third-party 
evaluators, such as university researchers or private research firms, to 
perform the evaluations, which in general measure self-reported changes 
in program participants’ behavior and attitudes related to sex and 
abstinence as indicators of program effectiveness. To obtain this 
information, the third-party evaluators have typically relied on surveys 
administered to program participants at the start of a program, its 
conclusion, and during a follow-up period anywhere from 3 months to 
almost 3 years after the conclusion. The third-party evaluations for 4 of the 



4tJ In order to estimate pregnancy rates among adolescents, states use data on both birth 
rates and abortions among adolescents. 

47 ACF does require grantees of the State Program to describe, in their grant applications, 
their plans for “tracking activities and measuring achievement” of their program goals and 
objectives. 
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6 states in our review have been completed as of February 2006, and the 
results of these studies have varied. 48 

Among the 4 states that have completed third-party evaluations, 3 states 
require the abstinence programs in their state to measure reported 
changes in participants’ behavior as an indicator of program 
effectiveness — both at the start of the program and after its completion. 
The 3 states require their programs to track participants’ reported 
incidence of sexual intercourse. In addition, 2 states require their 
programs to track biological outcomes, such as pregnancies, births, or 
STDs. In addition, 6 of the 10 states in our review require their programs to 
track participants’ attitudes about abstinence and sex, such as the number 
of participants who make pledges to remain abstinent. 

Some states also provide technical assistance to the abstinence-until- 
marriage programs they support in their state. This assistance is designed 
to help programs evaluate and improve their effectiveness. Officials from 5 
of the 10 states in our review either told us or provided documentation 
that they provide technical assistance on evaluations to abstinence 
programs in their state. One state official said that the abstinence-until- 
marriage programs supported by the state were found to be ill-prepared to 
conduct evaluations themselves, and that she now requires these programs 
to dedicate a portion of their grants to contract with a third-party or state 
evaluator to assist them in program-level evaluations. Officials from 
another state told us that they contract with a private organization of 
public health professionals in order to provide evaluation consultation and 
technical assistance for the abstinence-until-marriage programs the state 
supports. 



48 See, for example, LeCroy & Milligan Associates, Inc., Abstinence Only Education 
Program.: Fifth Year Evaluation Report, a report prepared for the Arizona Department of 
Health Services, 2003; Patricia Goodson et al., Abstinence Education Evaluation: Phase 6, 
a report prepared for the Texas Department of State Health Services, 2005; MGT of 
America, Evaluation of Georgia Abstinence Education Programs Funded Under Title V, 
Section 510, a report prepared for the Georgia Department of Human Resources, 2005; 
Thomas E. Smith, It’s G'reat to Wait: An Interim. Evaluation, a report prepared for the 
Florida Department of Health, 2001. 
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ASPE, CDC, and NIH Are 
Funding Studies Designed to 
Assess the Effectiveness of 
Abstinence-until-Marriage 
Education Programs 



In addition to ACF and OPA, other organizational units within HHS have 
made efforts to assess the effectiveness of abstinence-untii-marriage 
education programs. ASPE is currently sponsoring a study of the 
Community-Based Program and a study of the State Program. 49 For the 
former program, ASPE has contracted with Abt Associates to help design 
the study, and an ASPE official told us that once the agency selects an 
appropriate design, it will competitively award a contract to conduct the 
study. 60 For the latter program, ASPE has contracted with Mathematica 
Policy Research, Inc. (Mathematica), which is in the process of examining 
the impact of five programs funded through the State Program on 
participants’ attitudes and behaviors related to abstinence and sex. 51 As of 
August 2006, Mathematica has published two reports on findings from its 
study — an interim report documenting the experiences of schools and 
communities that receive abstinence-untii-marriage education funding, 
and a report on the first-year impacts of selected state abstinence-untii- 
marriage education programs. 62,63 Mathematica’s final report, which has not 
been completed, will examine the impact of the State Program on 
behavioral outcomes, including abstinence, sexual activity, risk of STDs, 



49 According to the House Appropriations Committee report accompanying the fiscal year 
2005 appropriation act for Labor, HHS, Education, and related agencies, the effectiveness 
of abstinence education programs should be determined by measures that include the 
prevention and reduction of adolescent pregnancies and STD infections, age at first sexual 
activity and intercourse, frequency of sexual activity and intercourse, and numbers who 
postpone sexual activity or intercourse through adolescence. See H.R. Rep. No. 108-636, at 
139-140 (2004). 

"According to ASPE officials, one factor that has contributed to delays in the initiation of 
this study is the difficulty in recruiting schools to participate. 

61 The five abstinence-untii-marriage education programs being studied are My Choice, My 
Future! in Powhatan, Virginia; ReCapturing the Vision in Miami, Florida; Teens in Control 
in Clarksdale, Mississippi; Families United to Prevent Teen Pregnancy in Milwaukee, 
Wisconsin; and Heritage Keepers in Edgefield, South Carolina. 

62 See B. Devaney et al., The Evaluation of Abstinence Education Programs Funded Under 
Title V Section 51 0: Interim Report, a report prepared for ASPE, 2002. 

63 See R. Maynard et al., First-Year Impacts of Four Title V, Section 510 Abstinence 
Education Programs, a report prepared for ASPE, 2005. Mathematica’s report on the first- 
year impacts of selected state abstinence-untii-marriage education programs focused on 
intermediate outcomes, including attitudes about abstinence, teen sex, and marriage; 
perceived consequences of teen and nonmarital sex; and expectations to abstain from 
sexual intercourse. 
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risk of pregnancy, and drug and alcohol use. 54 An ASPE official told us that 
the agency expects a final report to be published in 2007. 

Like ASPE, CDC has made its own effort to assess the effectiveness of 
abstinence-until-marriage education. CDC is sponsoring a study to 
evaluate the effectiveness of two middle school curricula — one that 
complies with abstinence education program requirements and one that 
teaches a combination of abstinence and contraceptive information and 
skills. 55 In CDC’s study, five middle schools chosen at random will receive 
a program consisting of abstinence-until-marriage education exclusively; 
five schools will receive comprehensive sex education, which also 
includes information on contraception; and five schools will be assigned to 
a control group. The study will examine the relative effectiveness of the 
programs on behavioral outcomes such as reported sexual risk behaviors 
and changes in attitudes related to abstinence and sex. CDC plans to 
recruit approximately 1,500 seventh grade students into its study and will 
follow them over a 2-year period. The agency expects to complete the 
study in 2009. 

NIH has funded studies comparing the effectiveness of education 
programs that focus only on abstinence with the effectiveness of sex 
education programs that teach both abstinence and information about 
contraception. As of August 2006, NIH is funding five studies, which in 
general are comparing the effects of these two types of programs on the 
sexual behavior and related attitudes among groups of either middle 
school or high school students. For example, in one NIH study, 
researchers are using groups of seventh and eighth grade adolescents to 
assess the impact of a variety of programs on, among other issues, 
adolescents’ reported sexual activities, knowledge, and beliefs. For this 
study, researchers are comparing these outcomes among students who 
received abstinence-until-marriage education; students who received a 
combination of abstinence and contraceptive education; and students who 
participated in a general health class, who serve as a comparison group. 



54 An impact evaluation assesses the net effect of a program by comparing program 
outcomes with an estimate of what would have happened in the absence of the program. 
This form of evaluation is employed when external factors are known to influence the 
program’s outcomes, in order to isolate the program’s contribution to achievement of its 
objectives. 

55 HHS officials told us that the two curricula being tested are intended to be comparable in 
length, intensity, and other characteristics. 
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NIH expects both this study and its other four studies to be competed in 
2006. 



Other Researchers Have Also 
Made Efforts to Assess the 
Effectiveness of Abstinence- 
until-Marriage Education 
Programs 



In addition to the efforts of researchers working on behalf of HHS and 
states, other researchers — such as those affiliated with universities and 
various advocacy groups — have made efforts to study the effectiveness of 
abstinence-until-marriage education programs. This work includes studies 
of the outcomes of individual programs and reviews of other studies on 
the effectiveness of individual abstinence-until-marriage education 
programs. In general, research studies on the effectiveness of individual 
abstinence-until-marriage education programs have examined the extent 
to which they changed participants’ demonstrated knowledge, declared 
intentions, and reported behavior related to sexual activity and abstinence. 
For example, some studies examined the impact of abstinence-until- 
marriage education programs on participants’ knowledge of concepts 
taught in the programs, as well as participants’ declared attitudes about 
abstinence and teen sex. Some studies examined the impact of these 
programs on such outcomes as participants’ declared commitment to 
abstain from sex until marriage, participants’ understanding of the 
potential consequences of having intercourse, and participants’ reported 
ability to resist pressures to engage in sexual activity. Some of the studies 
we reviewed examined the impact of abstinence-until-marriage programs 
on participants’ sexual behavior, as measured, for example, by the 
proportion of participants who reported having had sexual intercourse and 
the frequency of sexual intercourse reported by participants. In general, 
the efforts to study and build a body of research on the effectiveness of 
most abstinence education programs have been under way for only a few 
years, in part because grants under the two programs that account for the 
largest portion of federal spending on abstinence education — the State 
Program and the Community-Based Program — were not awarded until 
1998 and 2001, respectively. 



Several Factors Limit the 
Conclusions That Can Be 
Drawn about the 
Effectiveness of 
Abstinence-until-Marriage 
Education Programs 



Most of the efforts of HHS, states, and other researchers to evaluate the 
effectiveness of abstinence-until-marriage education programs included in 
our review have not met certain minimum criteria that experts have 
concluded are necessary in order for assessments of program 
effectiveness to be scientifically valid. For example, most of the efforts 
included in our review did not include experimental or quasi-exp erimental 
designs, nor did they measure behavioral or biological outcomes. In 
addition, the results of some assessment efforts that meet the criteria of a 
scientifically valid assessment have varied, and two key studies that meet 
these criteria have not yet been completed. 
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Experts Have Developed 
Criteria to Evaluate Efforts to 
Assess Abstinenee-until- 
Marriage Education Programs 



In an effort to better assess the merits of the studies that have been 
conducted on the effectiveness of sexual health programs — including 
abstinence-until-marriage education programs — scientific experts have 
developed criteria that can be used to gauge the scientific rigor of these 
evaluations. For example, in 2001, the National Campaign to Prevent Teen 
Pregnancy — an organization focused on reducing teen pregnancy — 
published a report by a panel of scientific experts that assessed the 
evidence reported on abstinence-until-marriage education programs in 
peer-reviewed journals and other literature. 56 The panel developed criteria 
that an evaluation of a program’s effectiveness must meet in order for the 
program’s results to be considered scientifically valid. In addition, in 2004, 
former U.S. Surgeon General David Satcher convened a panel of experts to 
discuss, among other things, best practices for evaluating the effectiveness 
of sexual health education programs — including abstinence-until-marriage 
education programs. 57 This panel published a report in 2006 that describes 
similar scientific criteria that assessments of program effectiveness need 
to meet in order for their results to be scientifically valid. Further, experts 
we interviewed agreed that these criteria are important for ensuring that 
the results of a study support valid conclusions. In general, these panels, 
as well as the experts we interviewed, agreed that scientifically valid 
studies of a program’s effectiveness should include the following 
characteristics: 

An experimental design that randomly assigns individuals or schools to 
either an intervention group or control group, or a quasi-experimental 
design that uses nonrandomly assigned but well-matched comparison 
groups. According to the panel of scientific experts convened by the 
National Campaign to Prevent Teen Pregnancy, experimental designs or 
quasi-experimental designs with well-matched comparison groups have at 
least three important strengths that are typically not found in other 
studies, such as those that use aggregated data: they evaluate specific 
programs with known characteristics, they can clearly distinguish between 
participants who did and did not receive an intervention, and they control 
for other factors that may affect study outcomes. Therefore, experimental 
and quasi-experimental study designs have a greater ability to assess the 



56 See Kirby. This panel included experts from NIH, the Medical Institute for Sexual Health, 
the Alan Guttmacher Institute, the Institute for Research and Evaluation, and various 
universities. 

57 See Satcher. This panel included experts from a variety of organizations, including the 
Medical Institute for Sexual Health, the Alan Guttmacher Institute, and the American 
Academy of Pediatrics. 
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Most Efforts to Assess the 
Effectiveness of Abstinence- 
until-Marriage Education 
Programs Have Not Used an 
Experimental or Quasi- 
Experimental Design 



causal impact of specific programs than other types of studies. 58 According 
to scientific experts, studies that include experimental or quasi- 
experimental designs should also collect follow-up data for a minimum 
number of months after subjects receive an intervention. 69 Experts 
reported that follow-up periods are important in order to identify the 
effects of a program that are not immediately apparent or to determine 
whether these effects diminish over time. In addition, experts have 
reported that studies should have a sample size of at least 100 individuals 
for study results to be considered scientifically valid. 60 

• Studies should assess or measure changes in biological outcomes or 
reported behaviors instead of a ttitudes or intentions. According to 
scientific experts, biological outcomes — such as pregnancy rates, birth 
rates, and STD rates — and reported behaviors — such as reported initiation 
and frequency of sexual activity — are better measures of the effectiveness 
of abstinence-until-marriage programs, because adolescent attitudes and 
intentions may or may not be indicative of actual behavior. For example, 
adolescents may report that they intend to abstain from sexual intercourse 
but may not actually do so. 

Many of the efforts by HHS, states, and other researchers that we 
identified in our review lack at least one of the characteristics of a 
scientifically valid study of program effectiveness. That is, most of the 
efforts to assess the effectiveness of these programs have not used 
experimental or quasi-experimental designs with sufficient follow-up 
periods and sample sizes to make their conclusions scientifically valid. For 



58 For example, experts have reported that the use of randomly assigned intervention and 
control groups is particularly important when assessing the effectiveness of abstinence- 
until-marriage programs because adolescents who voluntarily participate in such programs 
may be self-selecting — that is, they may be more willing to accept the principles of — and be 
influenced by — such programs when compared with other adolescents. Mathematica’s 
interim report on the evaluation of the State Program noted that selection bias can 
“seriously undermine the credibility” of study results. 

59 For example, one expert reported that studies assessing program effectiveness should 
obtain information on participants for at least 3 months after the conclusion of a program 
when they are measuring behaviors that can change quickly, such as frequency of sex. For 
behaviors or outcomes that change less quickly, such as initiation of sex or pregnancy 
rates, infonnation on participants should be collected for at least 6 months after the 
conclusion of a program. 

60 The panel of experts convened by the National Campaign to Prevent Teen Pregnancy 
agreed that large sample sizes are necessary to determine the magnitude of any discemable 
program effect and to ensure that results of any study of effectiveness are statistically 
valid. 
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example, ACF — and before it, HRSA — used, according to ACF officials, 
grantee reporting on outcomes in order to monitor grantees’ performance, 
target training and technical assistance, and help grantees improve service 
delivery. However, because the outcomes reported by grantees have not 
been produced through experimentally or quasi-exp erimentally designed 
studies, such information cannot be causally attributed to any particular 
abstinence-until-marriage education program. While ACF requires its fiscal 
year 2006 grantees of the Community-Based Program to contract with 
third-party evaluators to select and monitor outcomes for their programs, 
ACF is not specifically requiring these grantees to use experimental or 
quasi-exp erimental designs. Therefore, it is not clear whether these 
evaluations will include such designs. Similarly, ACF’s use of national data 
on adolescent behavior and birth rates to assess its State and Community- 
Based Programs is of limited value because these data do not distinguish 
between those who participated in abstinence-until-marriage education 
programs and those who did not. Consequently, these national data sets, 
which represent state-reported vital statistics and a nationwide survey of 
high school students, cannot be used to causally link declines in birth rates 
and adolescent sexual activity to the effects of specific abstinence-until- 
marriage education programs. 61 

Similarly, the efforts we identified by states and researchers to assess the 
effectiveness of abstinence-until-marriage education programs often did 
not include experimental or quasi-experimental designs. None of the state 
evaluations we reviewed that have been completed included randomly 
assigned control groups. For instance, one state evaluation that we 
reviewed only included students who volunteered to participate in the 
study. This evaluation report stated that the absence of a randomly 
assigned control group in the evaluation did not allow the evaluators to 
determine whether observed changes in participants’ reported sexual 
behavior — as indicated through surveys administered at the beginning and 
end of a program — could be attributed to the abstinence-until-marriage 
education program. 62 Similarly, some of the journal articles that we 
reviewed described studies to assess the effectiveness of abstinence-until- 



61 In addition, according to ACF and CDC officials, it is difficult to draw conclusions from 
national data sets about the effectiveness of abstinence-until-marriage education programs 
because the national survey questions used to produce these data often do not identify the 
specific type of program or intervention survey respondents may have participated in or 
received. 

62 See Goodson et al. 
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marriage programs that did not include experimental or quasi- 
experimental designs needed to support scientifically valid conclusions 
about the programs’ effectiveness. In these studies, researchers 
administered questionnaires to study participants before and after they 
completed an abstinence-until-marriage education program and assessed 
the extent to which the responses of participants changed. 63 These studies 
did not compare the responses of study participants with a group that did 
not participate in an abstinence-until-marriage education program. In 
addition, some of the studies used insufficient follow-up periods, thereby 
limiting the conclusions that can be drawn about the effectiveness of the 
abstinence-until-marriage education programs being studied. For example, 
two journal articles that we reviewed described studies that measured the 
effectiveness of abstinence-until-marriage programs in delaying the 
initiation of sexual activity from 1 to 2 months after completion of the 
program. 64 Scientific experts consider this follow-up period too short to 
assess whether the programs had a valid effect. 

According to scientific experts, HHS, states, and other researchers face a 
number of challenges in designing experimental or quasi-exp erimental 
studies of program effectiveness. According to these experts, experimental 
or quasi-experimental studies can be expensive and time-consuming to 
carry out, and many grantees of abstinence-until-marriage education 
programs have insufficient time and funding to support these types of 
studies. Moreover, it can be difficult for researchers assessing abstinence- 
until-marriage education programs to convince school districts to 
participate in randomized intervention and control groups, in part because 
of sensitivities to surveying attitudes, intentions, and behaviors related to 
abstinence and sex. For example, in a third-party evaluation of its 
program, one grantee of the State Program originally planned to 
administer follow-up surveys 1 year after participants finished their 
abstinence education program, but the evaluators decided not to conduct 
this follow-up because of confidentiality concerns and the difficulty of 
locating students. In addition, the contractors hired to design ASPE’s 



63 See, for example, S. M. Fitzgerald et al., “Effectiveness of the Responsible Social Values 
Program for 6th Grade Students in One Rural School District,” Psychological Reports, vol. 
91 (2002), and J. E. Barnett and C. S. Hurst, “Abstinence Education for Rural Youth: An 
Evaluation of the Life’s Walk Program, “ The Journal of School Health, vol. 73, no. 7 (2003). 

64 See, for example, D. A. Zanis, “Use of a Sexual Abstinence Only Curriculum with Sexually 
Active Youths,” Children & Schools, vol. 27, no. 1 (2006), and G. Denny et al., “An 
Evaluation of An Abstinence Education Curriculum Series: Sex Can Wait, ” American 
Journal of Health Behavior, vol. 26, no. 5 (2002). 



Page 35 



GAO-07-87 Abstinence Education 




Most Efforts to Assess the 
Effectiveness of Abstinence- 
until-Marriage Education 
Programs Have Not Measured 
Behavioral or Biological 
Outcomes 



study of the effectiveness of the Community-Based Program have reported 
difficulties finding school districts that are willing to participate in 
randomly assigned intervention and control groups receiving either 
abstinence-until-marriage education or comprehensive sex education. An 
ASPE official told us that although a “randomized approach” is the best 
design for assessing the effectiveness of a program, the approach is also 
the most difficult to conduct. 

Another factor that limits the conclusions that can be drawn about the 
effectiveness of abstinence-until-marriage education programs is the fact 
that most efforts in our review to study the effectiveness of these 
programs did not measure changes in behavioral or biological outcomes 
among participants. Instead, most of the efforts we identify in our review 
used reported intentions and attitudes in order to assess the effectiveness 
of abstinence-until-marriage programs. For example, neither ACF’s 
community-based grantees nor OPA’s AFL grantees are required to report 
on behavioral or biological outcomes, such as rates of intercourse or 
pregnancy. Similarly, the journal articles we reviewed were more likely to 
use reported attitudes and intentions — such as study participants’ reported 
attitudes about premarital sexual activity or their reported intentions to 
remain abstinent until marriage — rather than their reported behaviors or 
biological outcomes to assess the effectiveness of abstinence-until- 
marriage programs. For example, in one journal article we reviewed, 
participants were asked to rate the likelihood that they would have sexual 
intercourse as unmarried teenagers; another journal article described a 
study in which participants rated the likelihood that they would have 
sexual intercourse in the next year, before finishing high school, and 
before marriage . 65 

Experts, as well as state and HHS officials, have reported that it can be 
difficult to obtain scientifically valid information on biological outcomes 
and sexual behaviors. Specifically, experts have reported that when 
measuring an abstinence-until-marriage education program’s affect on 
biological outcomes — such as reducing pregnancy or birth rates — it is 
necessary to have large sample sizes in order to determine whether a small 
change in biological outcomes is the result of the abstinence-until- 



65 See, for example, L. Sather and K. Zinn, “Effects of Abstinence-Only Education on 
Adolescent Attitudes and Values Concerning Premarital Sexual Intercourse, “ Family and 
Community Health, vol. 25, no. 2 (2002), and G. Denny, M. Young, and C. E. Spear, “An 
Evaluation of the Sex Can Wait Abstinence Education Curriculum Series,” American 
Journal of Health Behavior, vol. 23, no. 2 (1999). 
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Results of Some Scientifically 
Valid Assessment Efforts Have 
Varied, and Other Key Studies 
Have Not Been Completed 



marriage education program. In addition, state and federal officials told us 
that they have experienced difficulties obtaining information on sexual 
behaviors because of the sensitive nature of the information they were 
trying to collect. For example, one state official told us that her state’s 
effort to evaluate abstinence-until-marriage education programs was only 
able to measure changes in participants’ reported attitudes, instead of 
behaviors, because the evaluators needed to obtain consent from the 
parents of the program participants in order to ask them about their 
sexual behavior. The state official explained that the requirement to obtain 
consent from parents raised issues of self-selection, and therefore state 
officials decided to ultimately halt the study and only report on the 
attitudes that they had measured. In another example, ACF’s fiscal year 
2006 budget justification reports that ACF has had some difficulty in 
obtaining reliable data from state grantees, in part because questions 
about teenage sexual behavior are sensitive. OPA officials also 
acknowledged that many communities will not allow grantees to ask 
program participants questions about their sexual behavior because the 
communities believe such questions are too intrusive. One OPA official 
said that such restrictions affect the agency’s ability to measure behavioral 
outcomes, explaining that OPA cannot measure what it cannot ask about. 

Among the assessment efforts we identified are some studies that meet the 
criteria of a scientifically valid effectiveness study. However, results of 
these studies have varied, and this limits the conclusions that can be 
drawn about the effectiveness of abstinence-until-marriage education 
programs. Some researchers have reported that abstinence-until-marriage 
education programs have resulted in adolescents reporting having less 
frequent sexual intercourse or fewer sexual partners. 66 For example, in one 
study of middle school students, participants in an abstinence-until- 
marriage education program who had sexual intercourse during the 
follow-up period were 60 percent less likely to report having two or more 
sexual partners when compared with their nonparticipant peers. 67 In 
contrast, other studies have reported that abstinence-until-marriage 
education programs did not affect the reported frequency of sexual 



66 See Borawski et al. See also T. L. St. Pierre et al., “A 27-Month Evaluation of a Sexual 
Activity Prevention Program in Boys & Girls Clubs Across the Nation,” Family Relations, 
vol. 44, no. 1 (1995). 

67 See Borawski et al. 
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intercourse or number of sexual partners. 68 For example, one study of 
middle school students found that participants of an abstinence-until- 
marriage program were not less likely than nonparticipants at the 1 year 
follow-up to report less frequent sexual intercourse or fewer sexual 
partners. 69 In addition to these varied findings, one study found that an 
abstinence-until-marriage program was effective in delaying the initiation 
of sexual intercourse in the short term but not long term. 70 Experts with 
whom we spoke emphasized that there are still too few scientifically valid 
studies completed to date that can be used to determine conclusively 
which, if any, abstinence-until-marriage programs are effective. 

Additionally, among the assessment efforts we identified are some studies 
that experts anticipate will meet the criteria of a scientifically valid 
effectiveness study but are not yet completed. One of these key studies is 
the final Mathematica report, contracted by ASPE, on the State Program. 71 
The final report was originally slated for publication in 2005, but an ASPE 
official stated that the final report has been delayed until 2007 so that 
researchers can extend the follow-up period to improve their response 
rate and the reliability of the information they collect. Another key study is 
CDC’s research on middle school programs, which is not expected to be 
completed until 2009. 

Experts and federal officials we interviewed stated that they expect the 
results of these two federally funded studies to add substantively to the 
body of research on the effectiveness of abstinence-until-marriage 
education programs. One expert with whom we spoke said that she 
expects the final Mathematica report on participants’ behaviors to provide 
the groundwork for the field. Another expert we interviewed stated that 



68 See N.G. Harrington et al., “Evaluation of the All Stars Character Education and Problem 
Behavior Prevention Program: Effects on Mediator and Outcome Variables for Middle 
School Students,” Health Education & Behavior, vol. 28, no. 5 (2001). See also Jemmott, 
Jemmott, and Fong. 

69 See Harrington et al. 

70 See Jemmott, Jemmott, and Fong and J. B. Jemmott III, L. S. Jemmott, and G. T. Fong, 
reply to letter to editor, Journal of the Am erican Medical Association, vol. 281, no. 16 
(1999), 1487. This study found that an abstinence-until-marriage program delayed the 
initiation of sexual intercourse at the 3-month follow-up period but not at 6 and 12 months. 

''According to several scientific experts, Mathematica’s study is an important one, in part 
because of its sound design: the study randomly assigns and compares control groups with 
groups receiving abstinence-until-marriage education and uses surveys to follow up with 
program participants for several months after their completion of a program. 
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the CDC study was very well-designed and she expects the results to 
contribute to the development of effective abstinence-until-marriage 
education curricula. 



Conclusions 



There have been various efforts — by HHS, states, and others — to assess 
the scientific accuracy of educational materials used in abstinence-until- 
marriage education programs and the effectiveness of these programs. 
However, efforts to evaluate both the accuracy and effectiveness of 
abstinence-until-marriage education programs have been, in various ways, 
limited. ACF, which administers the two programs that account for the 
largest portion of federal spending on abstinence-until-marriage education, 
does not review or require its grantees to review program materials for 
scientific accuracy. In addition, not all grantees of the State Program have 
chosen to review their materials. Because of these limitations, ACF cannot 
be assured that the materials used in its State and Community-Based 
Programs are accurate. Moreover, OPA, which reviews all grantees’ 
proposed abstinence-until-marriage educational materials, and states that 
review educational materials have found inaccuracies in some educational 
materials used by abstinence-until-marriage programs. 

Similarly, most of the efforts described in our review to assess the 
effectiveness of abstinence-until-marriage programs have not met 
minimum scientific criteria needed to draw valid conclusions about their 
effectiveness. Specifically, most efforts by agencies, states, and other 
researchers have not included experimental or quasi-experimental designs 
that can establish whether changes in behaviors or biological outcomes 
can be causally linked to specific abstinence-until-marriage education 
programs. While these types of studies are time-consuming and expensive, 
experts said that they are the only definitive way to draw valid conclusions 
about the effectiveness of these programs. In addition, among the 
assessment efforts we identified are some studies funded by HHS that 
experts anticipate will meet the criteria of a scientifically valid 
effectiveness study but are not yet completed. When completed, these 
HHS-funded studies may add substantively to the body of research on the 
effectiveness of abstinence-until-marriage education programs. 
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Recommendation for 
Executive Action 



To address concerns about the scientific accuracy of materials used in 
abstinence-until-marriage education programs, we recommend that the 
Secretary of HHS develop procedures to help assure the accuracy of such 
materials used in the State and Community-Based Programs. To help 
provide such assurances, the Secretary could consider alternatives such as 
(1) extending the approach currently used by OPA to review the scientific 
accuracy of the factual statements included in abstinence-until-marriage 
education to materials used by grantees of ACF’s Community-Based 
Program and requiring grantees of ACF’s State Program to conduct such 
reviews or (2) requiring grantees of both programs to sign written 
assurances in their grant applications that the materials they propose 
using are accurate. 



Agency Comments 
and Our Evaluation 



HHS provided written comments on a draft of this report. (See app. III.) 

In its written comments, HHS stated that it will consider requiring grantees 
of both ACF programs to sign written assurances in grant applications that 
the materials they use are accurate. Regarding accuracy, HHS’s written 
comments also noted that all applicants for federal assistance attest on the 
application form-Standard Form 424-that all data in their applications are 
“true and correct,” and that in the view of HHS, this applies to information 
presented in curricula funded by federal grants. However, as we stated in 
the draft report, grantees of the State Program are not required to submit 
curricula as a part of their applications; therefore, the attestation in 
Standard Form 424 would not apply to curricula used by those grantees. In 
addition, as stated in the draft report, some states have reviewed materials 
used in abstinence-until-marriage education programs, but these reviews 
occurred after they received funding from ACF. Further, while grantees of 
the Community-Based Program were required to submit copies of their 
curricula and a Standard Form 424 in fiscal year 2006 as part of their 
applications, none of the materials specifically require an assurance of 
scientific accuracy. Further, OPA and states have found inaccuracies in 
educational materials used by abstinence-until-marriage programs. 

HHS’s written comments also stated that ACF requires that curricula 
conform to HHS’s standards grounded in scientific literature. HHS’s 
comments refer to the curriculum standards for this program that detail 
what types of information must be included in abstinence-until-marriage 
curricula, and the comments stated that the curricula must provide 
supporting references for this information. Further, HHS’s comments 
stated that ACF staff review the curricula to ensure compliance with these 
standards. The draft report stated this. However, a requirement that 
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curricula include certain types of information does not necessarily ensure 
the accuracy of the scientific facts included in the abstinence-until- 
marriage materials. For example, while education materials may include 
information on failure rates associated with contraceptives or STD 
infections, this information may be outdated or otherwise inaccurate or 
incomplete. HHS’s written comments also stated that if it finds inaccurate 
statements during the review process or at any time during the grant 
period, ACF works with grantees to take corrective action. To ensure 
completeness, we have added this statement to the report. Further, HHS 
stated that 2 inaccuracies cited in the draft report had been corrected 
before our work began. We believe HHS is referring to inaccuracies 
identified by OPA during its review of materials for scientific accuracy and 
this reinforces the need for review of materials used by ACF’s grantees. 

As HHS noted in its written comments, we did not define the term 
scientific accuracy. HHS stated that it disagreed with certain findings of 
the report because it was difficult to precisely determine the criteria we 
employed in making the recommendation as to scientific accuracy. As we 
stated in the scope and methodology section of the draft report, the 
objective of our work was to focus on efforts by HHS and states to review 
the accuracy of scientific facts included in abstinence-until-marriage 
education materials. Performing an independent assessment of the criteria 
used by these entities to determine the scientific accuracy of education 
materials or the quality of the reviews was beyond the scope of the work. 

Regarding effectiveness, HHS’s written comments also described a 
number of actions it is taking to determine program effectiveness and 
improve the quality of programs and research. Specifically, HHS’s 
comments described (1) studies undertaken or funded by ASPE, CDC, and 
NIH; (2) technical assistance provided by OPA and ACF; (3) grantee 
evaluation requirements; and (4) ACF and OPA requirements for the 
amount of grant funds to be spent on evaluations. All of this information 
was included in our draft report. HHS’s comments also described a new 
effort funded by ACF and ASPE that is designed to build capacity for 
quality research in the field of abstinence education. We added 
information on this effort to the report. HHS’s written comments also 
describe evaluations that resulted from an Abstinence Education 
Evaluation Conference sponsored by ACF and OPA. While this conference 
was described in the draft report, we added more detail regarding the 
content of the conference. HHS’s written comments also describe OPA’s 
efforts to assess the effectiveness of the AFL Program. We had included 
this information in the draft report. 
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HHS’s written comments stated that it may be too soon to draw 
conclusions about the effectiveness of ACF’s and OPA’s programs, in part, 
because key studies have not been completed. We agree and discussed this 
in the draft report. As we noted in the draft report, key studies funded by 
HHS that experts anticipate will meet the criteria of a scientifically valid 
effectiveness study are not yet completed, but when completed these HHS 
funded studies may add substantively to the body of research on the 
effectiveness of abstinence-until-marriage education programs. 

In addition, the comments stated that having an inadequate amount of 
scientifically valid and conclusive evaluation studies is not unique to 
abstinence-until-marriage education programs, and a recent ASPE review 
of comprehensive sex education programs found mixed results on their 
effectiveness. However, the scope of our report was focused on 
abstinence-until-marriage education programs, and we did not review 
comprehensive sex education programs or make any comparisons 
between the two types of programs. 

HHS also provided technical comments, which we incorporated into the 
report as appropriate. 



As agreed with your office, unless you publicly announce its contents 
earlier, we plan no further distribution of this report until 30 days after its 
date. We will then send copies of this report to the Secretary of HHS and 
to other interested parties. In addition, this report is available at no charge 
on GAO’s Web site at http://www.gao.gov. We will also make copies 
available to others upon request. 

If you or your staff have any questions about this report, please call me at 
(202) 512-3407 or crossem@gao.gov. Contact points for our Offices of 
Congressional Relations and Public Affairs may be found on the last page 
of this report. GAO staff who made major contributions to this report are 
listed in appendix IV. 




Marcia Crosse 
Director, Health Care 
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Appendix I: HRSA’s Technical Assistance 
Contract for Abstinence Education 



The Health Resources and Services Administration (HRSA) awarded a 
contract to the National Abstinence Clearinghouse (NAC) in 2002 to 
provide assistance with its Community-Based Abstinence Education 
Program (Community-Based Program) and Abstinence Education Program 
(State Program). 1 NAC is a nonprofit educational organization whose 
mission is to promote the appreciation for and practice of sexual 
abstinence until marriage through the distribution of age appropriate, 
factual, and medically referenced materials. The purpose of the contract 
was (1) to develop national criteria for the review of abstinence-until- 
marriage educational materials and to create a directory of approved 
materials; (2) to provide medical accuracy training to grantees; and (3) to 
provide technical support to grantees, such as assistance with program 
evaluation. 2 

We are reporting on the steps that HRSA took to award the contract to 
NAC in response to concerns that have been raised by a congressional 
requester. In general, these concerns centered on the extent to which the 
selection process was competitive and whether HRSA identified the 
potential for an organizational conflict of interest. 

HRSA awarded the contract to address three concerns it had with the 
Community-Based Program during 2001, the first year of its 
implementation. First, HRSA officials needed guidance to determine 
whether abstinence-until-marriage education materials conformed to the 



'The administration of this contract was transferred from HRSA to the Administration of 
Children and Families (ACF) in May 2005. 

2 The contract resulted in the development of criteria for reviewing abstinence-until- 
marriage educational materials, and ACF included these criteria in the fiscal year 2006 
program announcement for the Community-Based Program. According to the 
announcement, ACF will evaluate grant applicants’ proposed educational materials to 
ensure compliance with the criteria. Medical accuracy training and technical support were 
provided to grantees as a result of the contract. According to an ACF official, a directory of 
approved abstinence-until-marriage educational materials was not completed. 
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Appendix I: HRSA’s Technical Assistance 
Contract for Abstinence Education 



definitional requirements of the Social Security Act. 3 Second, many 
grantees lacked the medical background and training to ensure that they 
would provide medically accurate, science-based information in their 
programs. Third, grantees also lacked experience with the technical 
management of federal grants, including how to conduct evaluations of 
their programs. 



HRSA Used Full and Open HRSA used full and open competition procedures to award the contract to 
Competition Procedures to NAC. In doing so, HRSA (1) publicly solicited proposals from potential 
Award the Contract to contractors; (2) conducted technical evaluations of both the original 

proposals and the revised proposals for those considered to be in the 
' 1 competitive range; and (3) determined that NAC’s proposal represented 

the best overall value to the government. This process, which took place 
from May 2002 through September 2002, resulted in HRSA awarding NAC 
the contract with a potential value of nearly $2.7 million. 

HRSA issued a notice on May 20, 2002, on the FedBizOpps Web site, the 
government point of entry for notifying potential contractors of federal 
contract opportunities, indicating its intent to publicly request proposals 
from prospective contractors in June 2002. 4 On June 20, 2002, HRSA 
posted the solicitation on the FedBizOpps Web site indicating that the 
abstinence contract would be awarded using full and open competition 
procedures, that is, all responsible prospective contractors would be 



3 Section 510(b)(2) of the Social Security Act defines abstinence education as an 
educational or motivational program that: A. has as its exclusive purpose, teaching the 
social, psychological, and health gains to be realized by abstaining from sexual activity; 

B. teaches abstinence from sexual activity outside marriage as the expected standard for 
all school age children; C. teaches that abstinence from sexual activity is the only certain 
way to avoid out-of-wedlock pregnancy, sexually transmitted diseases, and other 
associated health problems; D. teaches that a mutually faithful monogamous relationship in 
context of marriage is the expected standard of human sexual activity; E. teaches that 
sexual activity outside of the context of marriage is likely to have harmful psychological 
and physical effects; F. teaches that bearing children out-of-wedlock is likely to have 
harmful consequences for the child, the child’s parents, and society; G. teaches young 
people how to reject sexual advances and how alcohol and drug use increases vulnerability 
to sexual advances; and H. teaches the importance of attaining self-sufficiency before 
engaging in sexual activity. 

4 The current FedBizOpps Web site address is http://www.fbo.gov / . Prior to October 1, 

2005, the Web site address was http://www.eus.gov/ . The solicitation number was 240- 
MCHB-012(02)-abg. 
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Appendix I: HRSA’s Technical Assistance 
Contract for Abstinence Education 



provided the opportunity to compete. 5 The solicitation, which was a 
Request for Proposals (RFP), described the contract objectives, which 
included (1) the development of national criteria for the review of 
abstinence-until-marriage educational materials and the development of a 
directory of approved materials; (2) the provision of medical accuracy 
training to grantees; and (3) the provision of technical support to grantees, 
such as assistance with program evaluation. The RFP stated that HRSA 
intended to award a cost-reimbursement contract with fixed fee for a 
1-year base period and 2 option years. 6 This was a best value procurement; 
that is, HRSA reserved the right in the RFP to select for award the 
proposal that HRSA determined offered the best value to the government, 
even if it did not offer the lowest cost. Further, the RFP stated that the 
technical evaluation of the prospective contractors’ proposals would 
receive paramount consideration in the selection of the contractor. 
According to the RFP, this evaluation would include an assessment of the 
prospective contractor’s technical approach, the organizational experience 
and expertise of the prospective contractor, the plans for personnel and 
management of the work, and the prospective contractor’s statement and 
understanding of the project purpose. Other factors, such as the estimated 
cost, past performance under other contracts for similar services, and the 
subcontracting plan would also be considered in the selection process. 

Five prospective contractors submitted proposals to HRSA by July 31, 

2002, when proposals were due. 

HRSA established a review committee to conduct the technical evaluation 
of the five proposals. This committee included three voting members and a 
nonvoting chairperson. The Director of HRSA’s Community-Based and 
State Programs and two analysts from other programs within the 
Department of Health and Human Services (HHS) served as the voting 
members, and the chairperson of the review committee was a project 
officer of HRSA’s Community-Based Program. The committee members 
conducted the technical evaluation of the proposals, according to the 



5 The Federal Acquisition Regulation (FAR) requires the contracting officer to ensure that 
prospective contractors are responsible. FAR § 9.103. A responsible source refers to a 
prospective contractor that has, among other things, adequate financial resources, the 
necessary experience and technical skills to perform the work of the contract, a 
satisfactory performance record, and the ability to meet the delivery schedule. FAR § 9.104- 
1 . 

6 Cost-reimbursement contracts are used only when uncertainties involved in contract 
performance do not permit costs to be estimated with sufficient accuracy to use any type 
of fixed-price contract. FAR § 16.301-2. 
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Contract for Abstinence Education 



criteria in the RFP, as described above. Three proposals with the highest 
technical scores were determined to be in the competitive range, 7 with 
NAC’s proposal receiving the highest technical score. HRSA requested in 
writing that the competitive range offerors address certain technical and 
cost issues and submit revised proposals to HRSA by September 17, 2002. 8 
For example, HRSA requested that one of the prospective contractors 
other than NAC clearly describe its proposed management of day-to-day 
tasks of the contract and provide justification for several labor and travel 
expenditures. HRSA did not have oral discussions with the competitive 
range offerors. HRSA’s review committee evaluated the revised proposals 
and again gave NAC’s revised proposal the highest technical score. 

Although NAC’s estimated cost was not the lowest among the proposals in 
the competitive range, HRSA determined that NAC had proposed a 
realistic cost estimate for the contract. Accordingly, and in light of the 
NAC proposal’s high technical rating and the RFP’s evaluation criteria 
giving paramount consideration to the technical evaluation, HRSA 
determined that NAC’s proposal represented the best value to the 
government. HRSA awarded a contract to NAC on September 27, 2002. 

The contract had a 1-year base period of performance with an estimated 
value of $854,681, and included 2 option years for a total potential value of 
$2,673,784. According to a HRSA official, this cost-reimbursement contract 
did not include a fee. All of the prospective contractors were made aware 
that a debriefing to explain the selection decision and contract award 
would be provided at their request. One prospective contractor requested 
and received a debriefing from HRSA. No protests were filed with the 
agency challenging the award of the contract to NAC. There were no bid 
protests filed with GAO. 9 



HRSA Identified No Actual 
or Potential Organizational 
Conflicts of Interest 



HRSA officials told us that they did not identify any actual or potential 
organizational conflicts of interest during the acquisition process. As 
defined in the Federal Acquisition Regulation (FAR), an organizational 
conflict of interest arises where 



'Based on the ratings of each proposal against all evaluation criteria, the contracting officer 
establishes a competitive range consisting of all of the most highly rated proposals. FAR § 
15.306(c)(1). 

8 HRSA officials, including an auditor, reviewed the cost proposals in the competitive range. 

9 GAO’s Office of General Counsel resolves disputes concerning awards of federal 
contracts, which are known as bid protests. 
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• because of other activities or relationships, a person is unable or 
potentially unable to provide impartial assistance or advice to the 
government; or 

• the person’s objectivity in performing the contract work is or might be 
otherwise impaired; or 

• a person has an unfair competitive advantage . 10 

An organizational conflict of interest may result when factors create an 
actual or potential conflict of interest during performance of a contract, or 
when the nature of the work to be performed under one contract creates 
an actual or potential conflict of interest involving a future acquisition . 11 
Under the FAR, contracting officers are required to analyze planned 
acquisitions to identify and evaluate potential organizational conflicts of 
interest as early in the acquisition process as possible, and to take steps to 
avoid, neutralize, or mitigate significant potential conflicts of interest 
before a contract is awarded . 12 

According to HRSA’s contracting officer, HRSA did not identify any actual 
or potential organizational conflicts of interest. In reaching this 
conclusion, the contracting officer told us that he reviewed the statement 
of work, including the background and objectives of the proposed 
contract, the stated purpose of the contact, the criteria established to 
evaluate the proposals, the past performance of the competitors, and 
NAC’s proposal. HRSA’s contracting officer also told us that he did not 
formally document his assessment of organizational conflict of interest . 13 



“FAR § 2.101. 
n FAR § 9.502(c). 

12 The FAR requires contracting officers to exercise common sense, good judgment, and 
sound discretion in determining whether a significant potential conflict of interest exists. 
FAR § 9.505. 

13 The FAR requires HRSA’s contracting officers to formally document their assessment 
only when a substantive issue concerning a potential organizational conflict of interest 
exists. FAR § 9.504(d). HHS acquisition regulations do not explicitly address the 
assessment of organizational conflict of interest. Therefore, FAR subpart 9.5 is the 
controlling regulation when HHS encounters an issue related to an organizational conflict 
of interest. 



Page 48 



GAO-07-87 Abstinence Education 





Appendix II: Methodology for Identifying and 
Reviewing Research Studies 



To identify research studies that examine the effectiveness of abstinence- 
until-marriage education programs among adolescents and young adults, 
we searched two reference database systems, PubMed and ProQuest. We 
used the following keywords to search for research studies that were 
published from January 1, 1998, through May 22, 2006: “virginity,” 
“abstinence education,” “abstinence and curriculum,” “abstinence only,” 
“teen pregnancy and prevention,” and “abstinence until marriage.” 1 We 
reviewed the research article titles that were generated from the PubMed 
and ProQuest searches and identified articles that appeared to focus on 
the evaluation of the effectiveness of abstinence-until-marriage education 
programs. In cases where we could not determine, based on the title, 
whether a study appeared to focus on an abstinence-until-marriage 
education program evaluation, we reviewed a summary of the article to 
obtain more information about the research study. We also examined 
previous summaries of the literature to identify additional research 
studies. 

We then selected research studies for inclusion in our literature review if 
they met three criteria. First, the study evaluated a group-based, 
abstinence-until-marriage education program. We did not select studies 
that evaluated one-on-one interactions, such as education programs 
focused exclusively on parent-child interactions, or that evaluated media 
campaigns. We reviewed the description of each education program and 
curriculum, as described in the study, to determine whether an abstinence- 
until-marriage education program was being evaluated. Education 
programs that were described as including detailed contraceptive 
information in their curricula, for example, were not classified as 
abstinence-until-marriage programs. Second, the study targeted 
adolescents and young adults in the United States, for example, by 
indicating that participants in the evaluation were high school or middle 
school students. Third, the study was a quantitative rather than a 
qualitative evaluation of an abstinence-until-marriage education program. 
We selected 13 research studies for inclusion in our literature review. 



'ACF first awarded grants under the State Program in fiscal year 1998. 
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We reviewed the selected research studies to obtain detailed information 
about their methodologies and outcome variables. For example, we 
determined whether each study used an experimental or quasi- 
experimental design and whether the outcome measures included 
attitudes, behavioral intentions, behaviors such as initiation of sexual 
intercourse, or a combination of these. 
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DEPARTMENTOF HEALTH & HUMAN SERVICES 



Office of the Assistant Secretary 
for Legislation 



Washington, D.C. 20201 



SEP 2 2 2006 



Ms. Marcia Crosse 
Director, Health Care 
U.S. Government Accountability Office 
Washington, DC 20548 

Dear Ms. Crosse: 

Enclosed are the Department’s comments on the U.S. Government Accountal ility Office’s 
(GAO) draft report entitled, “ABSTINENCE EDUCATION: Efforts to Assess the Accuracy and 
Effectiveness of Federally Funded Programs” (GAO-06-984), before its publication. 

The Department provided several technical comments directly to your staff. 

These comments represent the tentative position of the Department and are subject to 
reevaluation when the final version of this report is received. 




Sincerely, / 

'X d U( 



Vincent J. Ventimiglia 
Assistant Secretary for Legislation 
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COMMENTS Oh THE DEPARTMENT OF HEALTH AND HUMAN SERVICES ON 
THE GOVERNMENT ACCOUNTABILITY OFFICE’S DRAFT REPORT ENTITLED. 
“ABSTINENCE EDUCATION: EFFORTS TO ASSESS THE ACCURACY AND 
EFFECTIVENESS OF FEDERALLY FUNDED PROGRAMS” fGAaf6-984j 

GAO Recommendations 

GAO recommends that to address concerns about the scientific accuracy of materials used in 
abstinence-until-marriage education programs, the Secretary of HHS develo]) procedures to help 
assure the accuracy of such materials used in the State and Community-based Programs. GAO 
further recommends that to help provide such assurances, the Secretary could consider 
alternatives such as (1) extending the approach currently used by OPA to review the scientific 
accuracy of the factual statements included in abstinence-until-marriage education materials used 
by grantees of ACF’s Community-based Programs and requiring grantees of ACF’s State 
Program to conduct such reviews, or (2) requiring grantees of both programs to sign written 
assurances in their grant applications that the materials they propose using are accurate. 

HHS Comments 

FIHS will consider requiring grantees of both ACF programs to sign written assurances in grant 
applications that the materials they use are accurate. However, as described below, ACF already 
requires applicants to submit curricula that conform to standards grounded in scientific data with 
their applications; and applicants for Federal grant assistance already assure the government that 
all data in their application, which includes data in curricula funded by a Federal grant, are true 
and correct. 

The report is divided into two main categories — scientific accuracy and effectiveness of 
abstinence-until-marriage programs. Comments are provided in that format. 



Scientific Accuracy 

Although GAO was requested to “assess the scientific accuracy of materials tsed in abstinence- 
until-marriage programs. . (page 3), GAO never defines the term “scientific accuracy” in the 
report. In fact, the report specifically recognizes (page 3) that it does not attempt to define or 
clarify the term. As such, it is difficult to precisely determine the criteria emoloyed by GAO in 
making the recommendations as to scientific accuracy. 

In part due to the lack of precision in the report’s use of the term “scientific accuracy,” but also 
due to the report’s incomplete depiction of how such materials are developed and reviewed, HHS 
does not agree with key claims of the report. One of the main claims of the report, for example, 
involves the often-repeated claim “ACF. . . does not review its grantees education materials for 
scientific accuracy, and does not require grantees.. . .to review their own materials for scientific 
accuracy. ...” (pages 5, 14,39). HHS disagrees for the following reasons: 

First, all applicants for Federal assistance attest in their applications that “all data in their 
applications are true and correct” (Standard Form 424, 

http://www.acf.hhs.gov/programs/ofs/forms.htm) . In the view of HHS, this applies to 
information presented in curricula funded by Federal grant funds. 

2 
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Second, ACF requires its abstinence-education grantees to submit educational curricula that 
conform to, and are thoroughly grounded in, scientific literature. Indeed, the Fiscal Year 2006 
Program Announcement for the Community-Based Abstinence Education Program, the largest of 
the three Federatly funded abstinence programs, required that successful applicants’ curricula 
conform to very specific, detailed curriculum standards (pages 5 through 13 of the program 
announcement, http://www.acf.hhs.gov/grants/open/HHS-2006-ACF-ACYF-AE-Q099.htmri . 
Each of these standards has robust hacking in scientific literature. Additionally, at least two of 
these standards discussed specific requirements by which successful applicants’ curricula would 
adhere to scientific accuracy. Namely, the guidance specifies that the curricula teach “the 
published failure rates associated with contraceptives relative to pregnancy prevention, including 
‘real use’ versus trial or ‘laboratory use,’ human error, product defect, teen use and possible side 
effects of contraceptives” (Theme Cl). Furthermore, the guidance for succe isful applicants’ 
curricula required that applicants’ curricula teach “the epidemiology of sexually transmitted 
diseases (STDs) in the U.S., e.g., infection rates, modes of transmission, existence of incurable 
and potentially fatal STDs” (Theme C2). Curricula must provide supporting references for this 
information. Upon submission, ACF staff reviews the application, including the curricula, to 
ensure compliance with these scientifically valid standards. Should we find inaccurate 
statements during the review process, or at any time during the grant project period, ACF works 
with the grantees to take corrective action, as they would do with any other aCF grantee. 

Increased grantee awareness of the scientific and medical facts relative to their delivery of 
abstinence education has been, and continues to be, an important area of focus for ACF. For this 
reason, ACF offers numerous training and technical assistance (T/TA) opportunities to all of its 
grantees on a number of program-related areas. In addition, as is the case with ACF’s grantees 
in other program areas, when ACF finds inaccuracies in a grantee’s information or when an 
inaccuracy is brought to the attention of ACF, whether during the review process or at any time 
during the project period, ACF requires and works with the grantee to have tie inaccuracy 
corrected as soon as possible. In the few cases of which ACF is aware of inaccuracies found to 
exist in abstinence materials, corrections were made once the grantee became aware of those 
inaccuracies. For example, the two inaccuracies cited in the GAO report were corrected before 
GAO even began working on the report. 

The GAO report presents an adequate assessment of OPA’s review of materials used in 
Adolescent Family Life (AFL) Abstinence Education Programs. There are, however, a few 
points about this review process that should be both clarified and highlighted. First, when OPA 
does find inaccuracies in education materials proposed for use in an AFL demonstration project, 
the grantee is notified and must make the specified correction(s) before OPA approves their use. 
Similarly, OPA grants include the condition, in each Notice of Grant Award issued for AFL 
demonstration projects, that all educational materials must be submitted to OPA for review and 
approval before they can be used in AFL projects. 

Aspects of the CDC review of materials for accuracy also warrant additional clarification. 
Specifically, all materials developed and/or used by national, State and local grantees are 
reviewed and approved by the grantees’ materials review panel. The review panel is a 
requirement of the CDC cooperative agreement. 

3 
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Effectiveness 

HHS is taking extensive action to determine program effectiveness and improve the quality of 
programs and research. Work is being done through the program offices, research agencies and 
through Congressionally mandated studies. These efforts include the following: 

• ASPE is overseeing a Congressionally mandated experimental evaluation of 
abstinence education programs that will examine behavioral outcomes over an 
extended period of time, and is in the process of designing another large-scale 
longitudinal study that will also include an experimental design and measure 
behavioral outcomes. 

• A new effort that is not included in the report is the development jf a Center for 
Research and Evaluation on Abstinence Education. This is a multi-year project 
funded by ACF and ASPE in FY 2006. It is designed to build capacity for quality 
research in the field of abstinence education. The main activities include a) 
conducting a needs assessment to identify gaps in abstinence-education evaluation 
and technical assistance needs; b) developing materials on abstins nce-education 
evaluation; c) delivering technical assistance and capacity-building activities related 
to program evaluation; and d) developing several research reports related to 
abstinence education. The goal is to stimulate greater collaboration and quality 
implementation and outcome-evaluation research of abstinence education programs. 

• CDC is undertaking a study to evaluate the effectiveness of two middle school 
curricula. The study will examine the relative effectiveness of pmgrams on 
behavioral outcomes. 

• NIH is currently funding several studies related to the effectivene >s of abstinence 
education, which are underway. 

• The two program offices (OPA and ACF) provide technical assistance and fund 
research activities related to evaluation. In addition, two of the giant programs (the 
AFL program and the Community-based Program) currently have evaluation 
requirements of grantees — OPA allows up to 25 percent of grant : unds awarded to be 
spent on evaluation and ACF is now requiring that a minimum of 15 percent of funds 
awarded to be spent on evaluation. 



4 
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Although GAO does acknowledge that “ACF has made efforts to assess the effectiveness of 
abstinence- until-marriage education programs. . (page 20), GAO fails to mention two key 
elements of these efforts. First, while GAO mentions the new evaluation requirement on 
Community-Based Abstinence Education (CBAE) grantees, there is no mention in the report that 
grantees are now required to spend at least 1 5 percent of their Federal funding on evaluation 
activities. Depending on the amount of the project, this amount could be up lo $450,000 over 
five years for each grant awarded. Never before have CBAE grantees been required to comply 
with such a significant commitment to evaluation and research activity. HHS believes that this 
requirement will dramatically improve its grantees’ ability to determine the effectiveness of the 
programs. The improvements made to this year’s guidance caused one peer- reviewed scientific 
medical journal to claim that “. ..a rigorous evaluation of project impact may now be possible." 1 
Second, the report omits any mention of the abstinence-education evaluation; from the first 
Abstinence Education Evaluation Conference sponsored by ACF and OPA. 2 The evaluations 
contained in this publication were peer-reviewed for presentation at the conference and conform 
to many of the standards published in the report (pages 3 1 through 33). ACF recommends that 
GAO insert language in the report reflecting these significant commitments to assessing program 
effectiveness. 

While GAO also generally acknowledges that “OPA has also made efforts to assess the 
effectiveness of the AFL Program” it would be helpful to specify what those efforts are since 
they, too, already conform to many of the standards published in the report. In particular, OPA 
has developed performance measures and requires projects to develop outcome objectives, 
conduct independent evaluations and use core evaluation instruments. 

Further, in response to the Office of Management and Budget’s (OMB) Prog-am Assessment 
Review Tool (PART), OPA has developed performance measures for their AFL program. Two 
of these performance measures apply to AFL prevention demonstrations. ALL Prevention Core 
Evaluation instrument data, aggregated across prevention projects, will be used to track them 
(Only the AFL projects, funded in fiscal year 2004 and later, are required to use the core 
instruments, and thus, it will require a few more funding cycles before tracking data are available 
for all projects.) 

Each AFL prevention-demonstration project is expected to develop specific and measurable 
outcome objectives, two of which should address the performance measures leveloped by OPA 
for prevention projects. While these objectives are intended more as a way to focus the 
individual project than as a way to evaluate it, projects are expected to tie their independent 
evaluations to their outcome objectives. 



1 Michael Young & Tina Penhollow, “The Impact of Abstinence Education: What Does the Research Say?” Journal 
of Health Education 37, no. 4 {July 1, 2006). 

2 Evaluating Abstinence Education Programs: Improving Implementation and Assessing In pact. Compiled 
manuscripts from “Strengthening Abstinence Education Programs through Scientific Evaluaion,” November 3-4 
2005. 
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All AFL demonstration projects are required, by statute, to conduct an independent evaluation of 
their project. The GAO report covers the many difficulties these evaluations face and OPA has 
taken steps, in the past few years, to improve the quality of its AFL evaluations. The limit of 
five percent of project funds for evaluation has been waived, pursuant to statutory authority, and 
beginning in fiscal year 2004, new projects can request up to 25 percent of p :oject funds for 
evaluation. Proposals considered for funding are held to commensurately more rigorous criteria 
with respect to their evaluation plan. Core evaluation instruments have been developed; all 
demonstration projects are encouraged to use them and, beginning in fiscal year 2004, all new 
demonstration projects funded by AFL were required to use them. The AFL program also 
provides, through expert consultants, evaluation technical assistance to all projects that require it. 

OPA has developed two sets of core evaluation instruments for the AFL projp-am baseline and 
follow-up for both prevention and care demonstrations to ensure that data, reflecting AFL 
legislative requirements, are captured in the independent evaluations. The in struments are 
intended as core only and additional instrumentation, addressing specific anc unique components 
of AFL demonstrations, can also be used in the individual independent evaluations. 

We note that because evaluation is a new requirement for ACF’s Abstinence Education Program 
and only recently intensified for OPA’s Abstinence Education Program, it miy be too soon to be 
drawing conclusions about their effectiveness. It would also be important to note that because 
youth arc typicaily 9-14 years of age in OPA’s Abstinence Education Progran and 12-1 8 years 
of age in ACF’s Abstinence Education Program, following youth in both programs takes time. 
That these large evaluations of the State and Community-based Programs arc still underway 
suggests that the conclusions drawn in the GAO report may be premature or speculative and may 
change significantly as new information becomes available. 

We agree with the report when it states that there are still too few scientifically valid studies 
which have been completed, to date, and which have been able to be used to determine 
conclusively which, if any, abstinence-until-inarriage programs have been ef fective. We would 
point out, however, that having an inadequate amount of scientifically valid and conclusive 
evaluation studies as of yet is not unique to abstinence until marriage education programs, A 
recent ASPE review of comprehensive sex education programs show that while some have a 
positive impact on some outcome measures, some have no impact and some have a negative 
impact. 

We believe that the answer to this challenge is not only a greater emphasis on evaluation among 
grantees, but also the conducting of a greater number of experimental programs generally. HHS 
would appreciate greater mention of both of these efforts. 

As HHS funds numerous abstinence demonstration projects across the countiy, and as we do so 
with an emphasis on evaluation, we believe that the field will mature to show the level of 
effectiveness and impact of these programs. Furthermore, HHS eagerly awaits the results of the 
first experimental trial referred to in the report, which will undoubtedly strengthen service 
delivery within the abstinence-education field, and benefit the youth who are choosing to remain 
abstinent in ever-increasing numbers. 

6 
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